The Anti-Aliasing Effect

By Richard Leadbetter Published 16 January, 2010

How developers are battling the jaggies.

Page 1 of 3

One of the most important elements in overall image consistency within console games has to the implementation of anti-aliasing - the process of smoothing edges and mitigating or eliminating the dreaded "jaggies". Both PS3 and Xbox 360 have AA technologies built into their respective GPUs, but as the console generation progresses it's becoming clear that for many games there are simply not enough system resources available to anti-alias images using traditional techniques. Developers are beginning to look beyond the hardware solutions and come up with their own methods. Some work well, some look awful, but it's all evidence of game-makers looking to eke out as much performance as they can from the current fixed architectures.

Before we continue, it's worth pointing out that aliasing doesn't just apply to dodgy-looking edges; it can also extend to other artifacts when rendering with texture and shaders, including moiré patterns and specular aliasing. However, the process of edge-detection and blending is the basis for this piece.

From a hardware perspective, edge-smoothing on Xbox 360 should theoretically be entirely "free", with the GPU doing the work for the developer. However, limitations in the on-die eDRAM memory stop that from being the case unless you opt for a sub-HD resolution (cases in point: Ninja Gaiden 2 and Modern Warfare 2). The memory requirement means that additional processing is required to implement anti-aliasing on 360 at 720p. For the PlayStation 3, running at a disadvantage in terms of bandwidth and memory, handling anti-aliasing is more difficult - hence this element being cut down in so many of the cross-platform games Digital Foundry looks at for the Face-Off comparison pieces.

PS3 is curious, however, in that it has hardware support for two widely used AA techniques. We've discussed MSAA already, but quincunx AA is the other most frequently implemented technique. Unique to the NVIDIA hardware, it uses approximately the same amount of resources as MSAA but produces superior edge-smoothing at the expense of adding a blur to the entire screen. The use of quincunx and the impact on overall image quality varies game by game - intricately detailed textures will suffer much more than a more flat, anime style of art. However, this Assassin's Creed comparison of 2x MSAA up against quincunx demonstrates both the edge-smoothing advantages and the detail blur.

'The Anti-Aliasing Effect' Screenshot ac-360
'The Anti-Aliasing Effect' Screenshot ac-ps3

Intricate detail is blurred with quincunx anti-aliasing, as seen in Assassin's Creed.

Colin McRae: DiRT 2 is an interesting example of how quincunx can be used to produce a look very close to the full-on 4x MSAA that can be extracted from the Xbox 360's Xenos GPU. The range of post-processing effects used in the EGO engine seem to suit quincunx as a "lighter" alternative to the 360's more precise but more memory-intensive solution. In short the game achieves some level of its realism from the motion blur employed, and in this case quincunx's less impressive side effects are integrated into the overall art scheme. Killzone 2 does the same thing.

'The Anti-Aliasing Effect' Screenshot dirt-360
'The Anti-Aliasing Effect' Screenshot dirt-ps3

The visual make-up of DiRT 2 makes the difference between MSAA and quincunx less impactful. The games look very close indeed.

The memory and bandwidth implications of the established hardware techniques can be seen as rather heavy, with game-makers looking to redirect that GPU effort elsewhere. That being the case, we have seen a number of customised anti-aliasing solutions in games such as Overlord II and Call of Juarez: Bound in Blood. We've also seen some fairly primitive edge-smoothing solutions that simply seek out the polygon edges and then blur them. Brutal Legend is a case in point.

'The Anti-Aliasing Effect' Screenshot ufc
'The Anti-Aliasing Effect' Screenshot coj
'The Anti-Aliasing Effect' Screenshot overlord
'The Anti-Aliasing Effect' Screenshot brutal

A growing number of games are using their own custom anti-aliasing solutions, including UFC, Call of Juarez: Bound in Blood, Overlord II and Brutual Legend.

"We have elected to use our own solution because hardware MSAA cannot be used while rendering using multiple render targets," says Call of Juarez lead shader programmer Maciej Jamrozik. "Another reason against going for hardware MSAA was the fact that it only smooths edges of geometry but not edges generated with alpha-test. The latter are prevalent in our titles, especially through the vegetation system. In addition to that, hardware MSAA requires extra memory which is a priceless resource especially on consoles. Finally, on Xbox 360 there’s the issue of buffer resolving which also reflects upon final performance."

Developer Techland went for its own custom anti-aliasing solution that copes with transparent objects, and has a lower impact on system resources while at the same time producing an effect fairly close to hardware MSAA. Not only that, but the support for multiple render targets means that their system would work in a deferred rendering engine, as seen in Killzone 2. Guerrilla Games actually added a more traditional forward renderer on top of its deferred tech partly to accommodate effects like anti-aliasing. The Call of Jaurez makers are in effect cutting out the middle man.

The recent release of The Saboteur by Pandemic revealed one of the most interesting uses of custom anti-aliasing seen to date. While it's said that the technique used was a simple case of finding edges and blurring them, the overall result is a huge increase in image quality over other examples. The algorithm runs on the PS3's SPUs, locating edges through luminance and then applying multiple passes of blending to give results that look absolutely phenomenal.

Page 1 of 3

Comments (53) Latest comment 2 years ago

Comments threads automatically close after 30 days, but please feel free to continue chatting on the forum!

  • Vroom #1 2 years ago

    Now that's some nerdy Saturday morning reading right there.

    Great stuff.

  • BBIAJ #2 2 years ago

    Awesome read on my birthday, cheers!

  • ChthonicEcho #3 2 years ago

    I always wondered how games manage to fake anti-aliasing with minimal performance costs. Dragon Age surprised me in that area. If I recall correctly, I don't have AA set, yet the game looks like it were set to 2x or even 4x.

  • dominalien #4 2 years ago

    Interesting article.

    I like my Sats because DF articles tend to come out :-)

  • Creasy #5 2 years ago

    @ChthonicEcho and it still looks like crap so no big deal there (art and technical wise)

  • capitanlt #6 2 years ago

    very interesting, thought the bad company stills looked much better when filtered. however i dont like the text blurring it gives. would there be a way to apply the hud after the filter?

  • hiddenranbir #7 2 years ago

    Doesn't running at a higher resolution help a bit?

  • Dizzy #8 2 years ago

    Console companies should make sure the next gen can do AA properly with near zero cost, this is a job for the GPU not for the devs.
    Edited by 1 at 16/01/10 @ 11:26

  • mrpon #9 2 years ago

    I bet Darren has laminated this article! ;)

  • mr_shoe_uk #10 2 years ago

    I may be the only person in the world, but I don't mind 'jaggies'. Playing some older things recently, they even lent them a kind of charm.

    The more vexing factor is when it causes thin lines of a dark against light colour, or vice versa, to flicker in and out, but this is still shown in the filitered versions of bad company 2 anyway.

  • Geordiemp #11 2 years ago

    The blurb is repeated by EG in almost every review :

    360 has GPU 'easy' AA but sub 720p when used iwth 10 MB EDRAM
    PS3 has CPU easy Qx but EG does not like the blur - not mad on it myself
    PS3 has SPU AA that does not impact performance if devs can be bothered
    The cheap edge filtering that works on static images is new, but looks limited on movement.

    3 pages summed in 4 lines. whats with the disagrees, just summem up the article..which does repeat allot in most reviews...
    Edited by 4 at 16/01/10 @ 16:08

  • Demiath #12 2 years ago

    This is really the angels-on-the-point-of-a-needle kind of video gaming scholasticism, but strangely compelling even to non-intelligent people like me...

  • schachmatt #13 2 years ago

    mr_shue: I don't mind jaggies in older games, especially Playstation 1 era, because producers were not only aware of the technical limitations, but even embraced it in incorporating that in the artwork and design.

    It's the total opposite of todays consoles, which promised pc-equivalent graphics they pathetically try to achieve, which just leads to the limitations being glaringly obvious plus problems with playability.

  • peanut80 #14 2 years ago

    "However, in the case of The Saboteur, the biggest issue is that the entire frame is processed before being passed to the video output - this means that the algorithm is detecting edges on the HUD, making the text look rather odd. "

    Is this an issue with using the SPUs instead of the GPU for anti-alisasing? Generally, I think you'd pass the render off to the render target before GUI elements were added, so that they're not included in anti-aliasing, but I'm guessing this usually is GPU-bound as a result of the programmable pipeline? I'm not really sure that should be the case.

  • local_celebrity #15 2 years ago

    Jaggies really are a minor quibble. What really gets my goat is the sudden slowdown you get when the console is overloaded. These always seem to happen at the most crucial moments, breaking the immersion and causing you to fluff things up.

  • Darren #16 2 years ago

    An interesting read.

    The custom AA used in the PS3 version of Saboteur looks astounding when put against the jaggy 360 version but I thought it was less noticable in the actual gameplay footage. The processed footage of other games makes them look higher resolution and less rough, in fact they look more like the PC versions in the case of Red Faction Guerilla and Mirror's Edge. Impressive.

  • Sharzam #17 2 years ago

    Iam a PC gamer primarly and i think that this can only be a good thing, because the PC approach is normally just chuck brute force at a problem. MSAA doesnt work, so what get a bigger GPU and force it. So with all this development on consoles to try and eeek out a way to get it can only be a good thing as it will have a knock on effect with PC development.

    Same goes with multi-core CPU development , PC games have been built with dual core for a while now where as using all 4 cores has been largly ignored with some exceptions (GTA4 for example) but if they are trying to get them to run on the consoles then will be optimised for multi-core.

  • Fake_Blood #18 2 years ago

    I don't really understand why they have to detect edges, is this because it's a post proces technique?
    When the cpu calculates geometry you basically have a wireframe with all the edges, can't you just blur every triangle/polygon?

  • bell_801 #19 2 years ago

    @ Geordiemp: Keep the FUD and your imagination to yourself. PS3 has way more sub-HD games than any console out there, minus the Wii (hello, old snake) and yet has none HW Solution to implement cost-free AA. Go, go Blurfest Games.

  • photoboy #20 2 years ago

    It will be very interesting to see if Microsoft include an eDRAM buffer on their next console. I think it's been the biggest reason most multi-platform games have looked better on the 360 this generation, so I'm sure they will want to replicate that success. With the advent of 3D they will probably need to double the size of whatever they would need for a normal single frame buffer image to accommodate both views of the scene. And ideally this time they'll want a buffer big enough to hold a full 720p image without needing tiling. Edit: Actually if we're talking next-gen we should be expecting 1080p frame buffers!

    I've often wondered why ATI haven't included eDRAM on their PC cards, if they used enough memory for a buffer size capable of holding a 1080p image with 8x anti-aliasing I would think it would be a significant advantage over Nvidia for them. I know it would boost costs, but when you've got the best performance, people will usually pay!
    Edited by 1 at 16/01/10 @ 16:44

  • Nithron #21 2 years ago

    I cannot believe how crap that pre-filtering shot of Uncharted looks. Every other shot i've seen of the game is fantastic, is that just a particularly bad-looking area?

  • WJF #22 2 years ago

    @hidden

    Being a completely ignorant soul for the programming side of gaming, but having just upgraded a PC to be able to run at a resolution higher than 1024x768 (Go me!) I can safely say that the higher the res, the more apparent jaggies become.

    At the lower res, it really doesn't matter if you have AA on as the number of pixels prevents seeing the 'crappy bits' when zoomed out, for instance, is so low that it doesn't matter. Once you start making the screen all crisp and smooth, the edges become a real annoyance.

    I've gone from 'why the hell would I want to slow down the game even further for a minor upgrade to graphics?' to 'God I wish I could boost the AA to 4x'.

  • KillerMonkey #23 2 years ago

    @WJF

    Huh?? Low resolutions make jaggies MORE apparent. Why? Because when you're running 640*480, the pixels are the size of bricks. At 1080p, they're better able to approximate diagonal lines/circles and stuff, leading to smoother edges. Now, if you mean the fact that LCDs upscale low res graphics and smother everything in massive blurry goo, then yes, the jaggies are less apparent. The problem is, EVERYTHING becomes less apparent.

  • des #24 2 years ago

    Sorry,the only true way to battle "jaggies" is to lift the resolution,but its the most expensive one.If next gen concentrates on 1920x1080,or close resolutions jaggies won't be a problem.Better spend those transistors on ai,high and stable frame rate,etc

  • Sunyavadin #25 2 years ago

    Quincunx makes me vomit. It makes HD games have softer edges by making them look like SD...
    Edited by 1 at 16/01/10 @ 15:31

  • Eraserhead #26 2 years ago

    I've always found the obsession with antialiasing a bit odd. It really doesn't bother me if there are a few ragged edges - you hardly notice them in most games if they're fast-paced or if you're running at 1920 x 1200.

    To me, by far the most important thing is resolution (on the PC, for instance), followed by high-quality texture filtering - has to be 16x anisotropic at high-quality! A road stretching off into the distance but blurring out after a couple of feet because of poor texture filtering looks *really* jarring.

  • Geordiemp #27 2 years ago

    To the bell

    Read the artcile, the article says the 360 can do MSAA through its EDRAM much easier than the Ps3 GPU but the free implemented MSAA on the GPU chip has to be sub HD to fit int the 10 MB frame. I just summed up the points. Of couse Xbox can do AA at any resolution but it takes moe power an not so 'free'.

    Its a benefit of the 360, and if you were not such a fanboy, you would see a potential solution to all consoles on next gen could be having say 20 MB of EDRAM, then the console could do MSAA at 720 with no impact.

    Try getting all consoes fanboy, would not be without Halo firefight, Crackdown (have 2 xboxes) or MGS4 / Uncharted 2 survival (have 2 Ps3s).

    Xbox multiplats are usually better with the EDRAM for graphics, Sony first party more impressive when heavy SPU's are used.

    Shame Xbox does not have so many SPU's (and M$ willingness to put first party effort in), and shame PS3 does not have a better GPU so easier for the 3rd party devs....

    Edited by 2 at 16/01/10 @ 16:09

  • cristoflanga #28 2 years ago

    I'm eager to play that "Brutual Legend" you're talking about.

  • Jayke #29 2 years ago

    great article

  • WJF #30 2 years ago

    @Killermonkey

    Hmm, now I come to think of it, it's probably because I can now run higher detail textures rather than the resolution itself that's causing the AA rage nowadays.

    Damn it! I maybe should think a little harder when submitting a reply. Oh well, it's not like I haven't been completely and utterly wrong before. :-D

  • freakzilla #31 2 years ago

    This is one of the reasons I'm hesitating about getting a new PC, to get the same graphics a PC uses a lot more electricity, and it seems as though they generally need to be more powerful to match console graphics.

    ...but the the transistors, heat-sinks and fans are so damn sexy.

  • Cirdain #32 2 years ago

    Metro 2033 has crazy Anti-analising

  • sangam #33 2 years ago

    اليكس|
    شباب اليوم|
    اخر الاخبار|
    اخبار|
    مدونة جحا|
    اخبار الجزائر|
    اخبار السعودية |
    اخبار الامارات |
    اخبار مصر|
    2lex|
    اسلامي 575;ت|
    افلام كرتون للاطفال|
    برامج|
    دروس|
    فيديوه 575;ت مضحكه|
    games|
    software|
    Audio & Multimedia|
    Graphic Apps|
    Web Development|
    System Utilities|
    Security & Privacy|
    Network & Internet|
    اخبار الاقتصاد المصرى|
    اخبار الثقافه المصرية|
    اخبار الحوادث المصرية|
    اخبار الرياضة المصرية|
    الاخبا 585; المصريه المحليه|
    قضايا مصريه ساخنة|
    اخبار الاقتصاد الجزائرى |
    اخبار التكنولو 80;يا الجزائري 07;|
    اخبار الرياضة الجزائري 77;|
    اخبار الثقافة الجزائري 77;|
    اخبار طبية جزائرية|
    اخبار الاقتصاد السعودى|
    اخبار الحوادث السعودية |
    اخبار الثقافة السعودية |
    اخبار سياسية|
    الاخبا 585; السعودية المحلية|
    اخبار الاقتصاد الامارات 09;|
    اخبار الثقافة الامارات 10;ه|
    اخبار المجتمع الامارات 09;|
    راى ودراسات|
    اخبار الرياضة الامارات 10;ه |
    احدث الاخبار|
    اسلامي 575;ت|
    برامج الكمبيوت 85;|
    المراه والاسرة|
    العاب|
    رياضة|
    منوعات|
    فيديوه 575;ت|
    مكتبة الموبايل
    برامج|
    اخبار جحا|
    اخبار الحوادث|
    اخبار العالم|

  • Nebula #34 2 years ago

    Cirdain: "Metro 2033 has crazy Anti-analising"

    Metro 2033 uses a kind of MLAA solution (morphological + other) for every frame to achieve smooth edges with compute shading. This can be used together with MSAA for icnredible jaggy reduction. It will work on DX10 Nvidia cards (assuming use of CUDA GPGPU for this) and probably on DX11 cards if they add DX11 support.

  • Nebula #35 2 years ago

    freakzilla: "This is one of the reasons I'm hesitating about getting a new PC, to get the same graphics a PC uses a lot more electricity, and it seems as though they generally need to be more powerful to match console graphics."

    A bit more efficiency out of hardware due to API overhead on PC and often much worse porting jobs than the console version jobs. However visuals still looks much better even with old 2006 year hardware like the 8800GTS and E6600 dual-core CPUs are far more powerful even with API overhead and API limitations. Allowing you much higher res, AA, anistropic filtering and that PC versions generally have higher detail settings. For many multiplatforms console version would equal low-med with some high bits.

    But as seen MLAA is interesting but still leaves a lot to be desired. While some surfaces are very smooth others are jaggied and transparencies looks worse. That would mean fooliage, fences, particle effects etc. Better than nothing with MLAA but far inferior to the uniform clean image you get with say 4xMSAA and 2-4xTSAA which is pretty standard and playable even on budget GPUs (80-100$) with almost all games with moderate resolutions.

    Edited by 3 at 17/01/10 @ 00:04

  • Hastur #36 2 years ago

    AA is for puritans imo. In the past, when 3D models consisted of fewer polygons, it could make a big difference. But nowadays you will only notice the jaggies under a microscope.

  • freakzilla #37 2 years ago

    @Nebula

    I guess I can't say with any authority whether PCs or consoles are more efficient. But it is widely accepted that PC devs don't usually put early as much effort into optimisation as console devs isn't it?

    And it seems pretty certain that a PC can't put out PS3 graphics while using the same amount of power.

    http://ne ws.cnet.com/8301-17938_105-1031...
    Edited by 1 at 17/01/10 @ 01:48

  • Nebula #38 2 years ago

    freakzilla : "I guess I can't say with any authority whether PCs or consoles are more efficient. But it is widely accepted that PC devs don't usually put early as much effort into optimisation as console devs isn't it?

    And it seems pretty certain that a PC can't put out PS3 graphics while using the same amount of power."

    Yes PC draws more Watt to function regarding a gaming PC when playing games vs PS3 slim. But one could also say PC gives you far more per Watt as in perfomance, features and functions! :p

  • hahayou #39 2 years ago

    It's impressive, and I didn't see what the problem with motion in the processed videos was. Is there some particular spot I should look at? By far the nastiest visual artefact I saw was the chainlink fences in the distance phasing in and out, something that was equally crap with and without edge-AA.

    @Fake_Blood: edge detection makes some sense - there's no point in blurring edges that you can't see because they're black-on-black or behind smoke or something.

  • monkeywithnoeyes #40 2 years ago

    The Saboteur examples just has the ps3 version looking darker and more blurred.. with textures more muted, much like other games that have used the ps3's "smoothing" techinique in replace of MSAA - i don't see any benefit in that.

    Also, there seems to be little benefit in it if it's only "pretty" when in still pics... it resulting in a broken up look during gameplay.

    I'm sure i'll get labelled a fanboy.. but i'm really not seeing the benefit of it..especially when you've actual game developers commenting on it, the saboteur footage is left with the textures looking more blurred over any noticable advantage, and theres nothing mentioned on comparrison of how the 360 and ps3 versions of the game runs - which i would assume important

  • Buran #41 2 years ago

    The lack of Super Sampling AA in almost any console game in this generation is both a testament of their limitations and a evidence of the superior technology displayed in the PC games and engines.

  • Murton #42 2 years ago

    To be perfectly honest jaggies don't bother me too much if the game is good, it's only when you have widespread jaggies and other quality issues, such as poor frame rate, screen tearing, hangs/crashes/deadstates etc. that is becomes a problem for me. Infamous for example is a mess of jaggies but it doesn't stop it from being a good quality gaming experience. Fallout 3 on PS3 was also an utter mess when it comes to edges, but it also suffered from a shockingly low frame rate, insanely frequent hangs/deadstates and many other issues which when taken together are quite difficult to really enjoy and begs the question of why Sony didn't fail its submission on grounds of poor build quality and bar its release until it was finished.

    Jagged edges, like so many flaws with modern games are for the most part, a question of effort. There are many effective techniques for combatting the jaggies these days, it's purely a matter of optimising your code enough to be able to use one of them without completely destroying the game in terms of build quality.

  • smelly #43 2 years ago

    (sarcasm)
    I dont know about you lot. But when i'm having fun enjoying an entertaining game.. Jaggies which i can only notice if i stand still and look at the scene in detail - REALLY bother me.
    (/sarcasm)

  • M_of_the_sys #44 2 years ago

    I never really noticed jaggies until I read this article. Now I'm sure I will see them all over the gaff! Damn you DF. Now I'll have to trade in my PS3 for a 360.

    On a side note, I don't know if anyone can answer this but when ever I look at comparison screen shots, why are the 360 shots always darker? For example, in the Saboteur screenshots for the 360 I couldn't even see any edges because of the shadows.

  • oreillymj #45 2 years ago

    I'm amazed that Saboteur has such hi-tech AA code in there. Perhaps the coders should have spent more time on the controls. The general feel of the game is just wrong, and the camera/gun aiming mechanic is toilet.
    The vehicle driving sections feel like GTA. Like driving on ice. The cars are so uncontrollable, I spent a lot of time on pavements mowing down pedestrians by accident.

  • anss123 #46 2 years ago

    How much I notice jaggies depend on the game. Rule of a thumb is "the higher the resolution and the better the graphics the more I notice the jaggies", the high resolution bit may seem odd but it's still true.

    I blame the uncanny valley effect.
    Edited by 1 at 18/01/10 @ 13:09

  • metalnut #47 2 years ago

    This article is totally wrong about needing to use post-processing tricks in the next generation. My PC's video card (top-end 3 years ago so fairly cheap now) can do 8xMSAA with coverage-sampled AA (alpha tested AA, for foliage) almost without breaking a sweat, and at resolutions exceeding 720p.

    All these post-process fakes for AA will be defunct in the next generation, they suck compared to proper sub-pixel AA and it's definitely doable in affordable hardware at 720p, just not on the ageing cut-down 5-year old GPUs in today's consoles.
    Edited by 1 at 18/01/10 @ 14:15

  • Les #48 2 years ago

    I've never been a fan of the soft look of AA TBH. Can't really be bothered by jaggies. What makes or breaks graphics for me is lighting and in most current games that aim for the realistic look it is absolutely terrible.

  • mkreku #49 2 years ago

    Jaggies? What jaggies?

    /strokes new Radeon 5870

  • neems #50 2 years ago

    Well I don't recommend running your 1600x1200 monitor at 1280x1024 yomomma, as they are different aspect ratios, it will make your graphics look squashed or stretched (I forget which). Try 1280x960 instead, that'll give you a proper 4:3 AR.

  • smelly #51 2 years ago

    >jaggies arent really noticable when ur moving,

    Agreed. Although if a game is fun, i'm usually too busy actually playing the game to notice any graphical glitches (Apart from every now and then)

  • zedzee #52 2 years ago

    "One of the most important elements in overall image consistency within console games has to [be] the implementation of anti-aliasing..."

    Not a grammatically good start to an otherwise interesting article.

    "game-makers looking to eke out as much performance as they can from the current fixed architectures."

    Surely if performance is what they want to "eke out", then adding AA won't help?! Something wrong with that statement...

  • deadfloppy #53 2 years ago

    Thats a very interesting and educating article but i have one question,

    Why is some people saying that with their super 30+" TVs they dont see any jaggies? Isnt according to the pictures and videos in this article how XBOX supposed to look?