How Shawn learned to stop worrying and love premultiplied alpha

How Shawn learned to stop worrying and love premultiplied alpha

  • Comments 17

I like to think I know what I'm doing with computer graphics, so I find it embarrassing that it took us four versions to get something as fundamental as alpha blending right.

To cut a long story short, I didn't properly understand alpha blending until way too recently :-)

(but in my defense, it seems like many other people don't understand it either)

To leave the long story long:

Graphics programming, especially for games, is an interesting mix of science, art, and Rube Goldberg contraption. Some things can be mathematically proven to have a single correct solution, so you'd be crazy to do them any other way. Other times we can achieve great results from ad-hoc hacks that only make sense for a specific game or piece of hardware.

When I started making games, the focus was very much on the hack side of things. Most projects began with being given a piece of hardware, and told to go off and make it do something cool. Few people had formal backgrounds in math or computer graphics.

Over time, the focus shifted toward properly understanding the theory behind what we were doing. Many projects began with being given a stack of SIGGRAPH papers, and told to go off and implement the techniques described therein. This often left those of us who grew up figuring things out by trial and error scrambling to relearn old tricks.

So how did I learn about alpha blending?

The first time I encountered the concept of blending colors was in college, where I wrote a game programming library called Allegro. Working with 8 bit palettized colors, this included a draw_trans_sprite() function which used a 64k lookup table to precompute every possible combination of two color values. This could achieve many varied effects just by changing what went into the table, but with only 256 colors to choose from, the quality of the results was, well, kinda dubious :-)

The first time I experienced proper hardware alpha blending was on the Nintendo 64, where we used a mixture of additive and interpolative blending, mostly for special effects. Textures were extremely small, usually just 4 bpp, and drawn by hand with Deluxe Paint. If we saw filtering or blending artifacts, we just asked the artist to change the offending texture.

The first time I ran into alpha blending problems was on Playstation II, where the artists used scanned photos as a source for creating 4 and 8 bpp palettized textures. Thanks to the unholy combination of filtering and interpolative alpha blending, we got ugly fringes around the edges of our tree billboards. I fixed this by using our equivalent of a custom content processor to automatically adjust the RGB of transparent palette entries, changing them to match the average color of the texture as a whole. Problem solved (or so I thought...)

The second time I ran into alpha blending problems was when we moved from Playstation II to the original Xbox, and switched from palettized textures to the more efficient DXT compression. DXT1 only supports transparent pixels where the RGB is zero, so my automatic fixup was no longer possible and the black fringes returned. I 'fixed' this by switching to DXT5 format wherever possible, plus using alpha test to skip drawing the problematic borders of textures for which DXT5 would use too much memory. This was a delicate balancing act. If the alpha test threshold was too high, the trees appeared jagged and hard edged, but it was too low, they had black borders. Somewhere in between, both problems still existed, but neither was too obvious. Problem solved? (kinda, but it made me feel dirty...)

The third time I ran into alpha blending problems was when I optimized particle rendering by drawing into a lower resolution rendertarget. Rendertarget composition is fundamentally impossible when using interpolative alpha, but I didn't know this at the time. So I sat down with pen and paper, did some algebra, and figured out the blend math necessary to get the result I wanted. It turns out that I independently reinvented premultiplied blending, but I failed to generalize this or realize how useful it could be for other things!

A few years later, I read Tom Forsyth's excellent article about premultiplied blending. "Aha!" quoth I. "So that's what that crazy thing I used for the particles is called. And wow, it seems pretty handy for other stuff too." Then I promptly forgot all about it.

A few more years later, I left Climax, left England, moved to Seattle, joined Microsoft, and started working on XNA. Having forgotten Tom's article, it never occurred to me (or I guess to anyone else here) that this might be something worth including in our API.

Shortly after we shipped XNA v1, I saw the first "wtf is up with these crazy borders around my sprite?" question on our forums. In the middle of typing up a lengthy "here's the laundry list of hacks I used to work around this problem in MotoGP" reply, I remembered, "wait, didn't Tom know a better way to fix this?"

After several years of getting this same question, pointing people toward Tom's article, and watching the ensuing confusion because it was too hard to do this with our API, we decided it was time to fix this mistake once and for all.

Enter Game Studio 4.0.

And they all lived happily ever after.

(touch wood)

Makes me wonder what else is out there that I don't know but should?

  • Cool post :)

    I never knew that you made allegro... how long did you work on that for?

  • I read this blog because you made Allegro.

  • I love these small story like parts in between the tech articles. I so recognize myself in there sometimes (most techies will).

    I have an equal story, but about developing database driven applications. (I was once asked to make one when I was 17, and all I had done was toying with VB6, after that one when I was just studying CS for 6 months, and one after a few years of study, can you gues the quality? :-) ).

  • With all this talk about premultiplied alpha lately, I can't help but "worry". :)

    Isn't PMA only practical for a limited range of (lightly colored / additively blended) things like clouds or fire?  I mean, black can't be drawn at all, colors wash out and turn transparent, etc...

    I think it's a great idea making PMA easier to use, but making it default seems so strange to me.  I hate repetitive/rude support questions, which is why I selfishly stopped supporting RocketDock, so I do sympathize... I just think PMA makes more sense as an option, otherwise your questions will just turn into "Why is my frickin' tree transparent?!? The alpha is 255 in Photoshop!"  And honestly, that's a valid complaint compared to inherent alpha/edge issues that are beyond the scope of what XNA can/should handle.

    Just my silly 2 cents... You did write Allegro, so you should probably just ignore me. ;)

  • Okay, I reread your posts on premultiplied alpha.

    For some reason I assumed the alpha channel was dropped so only additive blending was possible.  At least I was right about one thing: I should be ignored. *hits head* :)

  • Just out of curiosity (I'm not an XNA user), is there any gotcha in using premultiplied alpha and sRGB colors ? Does XNA content processor handle this case ?

  • Pre-multiplied alpha is cool, but I wouldnt go as far as shawn and give the impression it is the solution to all alpha blending problems:-)

    There are plenty of other alpha tricks, eg fading alpha based on depth, fading based on depth to object in depth buffer(ie soft particles), sorting meshes into a set of fixed directions(grass), sorting into shells(eg hair layers) etc etc

    It is a shame that standard graphics texts do not cover this type of thing very well(I cant even remember any references from Foley and Van Dam, or the more recent real time rendering)

    I too learned about pre multiplied alpha(and lots of other neat tricks) from posts by TomF etc...

  • > I never knew that you made allegro... how long did you work on that for?

    Pretty much the whole time I was in college, when I should have been studying for my degree, I was writing Allegro code instead :-)

    But hey, that's how I learned to program, so I figure it paid off in the end!

    After I graduated and got a job, I maintained it for a couple more years, mostly just integrating changes from other people rather than writing huge new things myself. Then I gradually phased out my involvement and handed the project over to other maintainers (who have been doing a great job keeping it going ever since).

  • > Isn't PMA only practical for a limited range of (lightly colored / additively blended) things like clouds or fire?  I mean, black can't be drawn at all, colors wash out and turn transparent, etc...

    I think you are confusing premultiplied with additive blending. Additive has all the characteristics you describe here, which makes it great for specific effects (fire, magic, glows) but useless for anything elsee.

    Premultiplied blending is a direct algebraic rearrangement of interpolative blending, so anything you can do with interpolative colors, you can do with premultiplied as well, just with less artifacts.

  • > is there any gotcha in using premultiplied alpha and sRGB colors ?

    Premultiplied and sRGB can work together, but (like pretty much anything involving sRGB formats) it does require some care to get the right results.

    However, XNA doesn't currently support sRGB at all.

  • > Pre-multiplied alpha is cool, but I wouldnt go as far as shawn and give the impression it is the solution to all alpha blending problems:-)

    I wouldn't go that far either! I just think premultiplied is a good default place to start (certainly much better that interpolative alpha).

    > There are plenty of other alpha tricks, eg fading alpha based on depth, fading based on depth to object in depth buffer(ie soft particles), sorting meshes into a set of fixed directions(grass), sorting into shells(eg hair layers) etc etc

    Oooh oooh - methinks you need to write articles about these techniques! That's an awesome list of blog content right there :-)

  • >>> Pre-multiplied alpha is cool, but I wouldnt go as far as shawn and give the impression it is the solution to all alpha blending problems:-)

    I wouldn't go that far either! I just think premultiplied is a good default place to start (certainly much better that interpolative alpha).

    <<<

    I guess often pre-multiplied alpha is touted as a way to avoid sorting etc. Which it isnt really, although often the results can be OK for some things.

    >>> There are plenty of other alpha tricks, eg fading alpha based on depth, fading based on depth to object in depth buffer(ie soft particles), sorting meshes into a set of fixed directions(grass), sorting into shells(eg hair layers) etc etc

    Oooh oooh - methinks you need to write articles about these techniques! That's an awesome list of blog content right there :-)

    <<<

    Well I tend to prefer writing code to blogging non stop:-P

    Anyhow I just wrote an entry on why the standard light attenuation function(D3D/OGL fixed function) sucks....

    Which is a good example of why even having only simple programmable shaders is a good thing.

  • i said it before and i'll say it again, investigate QFloats for premultiplied alpha. brings the premultiplied component down to 1 byte (q = 7) - as well as being faster during composition.  it does have a (impercievable) decrease in quality.

    real trick is to get the gpu to accept it.

  • So whats a QFloat? The alpha is normally 8bits anyway, so what do they gain? Is it some kind of compression for HDR data perhaps?

    Perhaps you could also write a blog entry about QFloats...? :-)

  • Some more while I'm at it:

    Texture processor:

    - Why is color key on the default?

    - Why is mip maps off the default?

Page 1 of 2 (17 items) 12
Leave a Comment
  • Please add 6 and 7 and type the answer here:
  • Post