Premultiplied alpha and image composition

Premultiplied alpha and image composition

  • Comments 6

From Wikipedia:

"Alpha compositing is the process of combining an image with a background to create the appearance of partial transparency. It is often useful to render image elements in separate passes, and then combine the resulting multiple 2D images into a single, final image in a process called compositing."

 

Example

I am making a Pong game. I start by clearing my background to CornflowerBlue (100, 149, 237). I then draw a translucent grey box (128, 128, 128, 128) at the top of the screen, which will form the background of my score display. With conventional alpha blending:

((128, 128, 128) * 0.5)  +  ((100, 149, 237) * (1 - 0.5))  =  (114, 138, 182)

Well and good. But over time, my overlays get more complex, including a map, radar, waypoint indicators, mission objectives, leaderboard rankings, etc. (what can I say, this is a complex Pong game :-)  To avoid redrawing so much stuff every frame, wouldn't it be cool if I could draw the overlays just once to a rendertarget, then copy the resulting static image over the top of my animating gameplay?

 

A Problem

Unfortunately, this doesn't work using conventional alpha blending:

Clear rendertarget to (0, 0, 0, 0)

Blend 50% grey into the rendertarget:

      ((128, 128, 128, 128) * 0.5)  +  ((0, 0, 0, 0) * (1 - 0.5))  =  (64, 64, 64, 64)

Clear backbuffer to CornflowerBlue (100, 149, 237)

Blend rendertarget over the backbuffer:

      ((64, 64, 64) * 0.25)  +  ((100, 149, 237) * (1 - 0.25))  =  (91, 127, 194)

Whoah! That's not even remotely the same result as before. The blend operation is happening twice: first when we draw into the rendertarget, then again when we draw the rendertarget into the backbuffer. Our alpha value ends up getting squared, so 0.5 becomes 0.25.

We could fix this by removing one of the blend operations:

  • Disable blending while drawing to the rendertarget
  • Or disable blending while drawing the rendertarget over the backbuffer

But these are rarely good solutions, since you most likely need blending in both places  (the overlays in my Pong game are built from many layers of alpha blended graphics, and the gameplay scene should be visible behind them).

 

Some Math

Given a series of drawing operations which blend over the top of each other:

result = a -> b -> c -> d

we would like to be able to group a set of calls from the middle of this sequence, storing their result in a rendertarget, then later replace that set of calls with the contents of the rendertarget:

tmp = b -> c

result = a -> tmp -> d

The problem is that this changes the order of the blend operations. What was:

result = blend(blend(blend(a, b), c), d)

becomes:

result = blend(blend(a, blend(b, c)), d)

Because conventional alpha blending is not associative, changing the order of evaluation changes the result.

 

The Solution

Premultiplied alpha blending is associative, so any number of blending operations can be grouped and reordered without affecting the final result.

To use premultiplied alpha, we make two changes:

  • Convert our source colors into premultiplied format, so translucent grey becomes (64, 64, 64, 128) rather than (128, 128, 128, 128)

  • Change RenderState.SourceBlend from Blend.SourceAlpha to Blend.One

Working through the same example as before:

Clear rendertarget to (0, 0, 0, 0)

Blend 50% grey into the rendertarget:

      (64, 64, 64, 128) +  ((0, 0, 0, 0) * (1 - 0.5))  =  (64, 64, 64, 128)

Clear backbuffer to CornflowerBlue (100, 149, 237)

Blend rendertarget over the backbuffer:

      (64, 64, 64) +  ((100, 149, 237) * (1 - 0.5))  =  (114, 138, 182)

Tada! That's the same result as when we were drawing everything directly to the backbuffer.

Premultiplied alpha r0x0rz.

 

Historical Aside

Alvy Ray Smith writes about the invention of the alpha channel and image composition. I find it interesting that this was invented in the 1970s, and fully grokked as of the classic Porter-Duff paper in 1984, yet here we are a quarter of a century later, still having a hard time making composition work right because for some reason premultiplied alpha never became as widely understood as it deserves to be.

  • But what will happen if at some step clamping will occur?

  • > But what will happen if at some step clamping will occur?

    I'm not sure what you mean?

    With any rendering operation, you need to make sure you use formats that have enough range to represent the data you are dealing with.

    Rendertarget composition or using premultiplied alpha does not change anything in this regard compared to if you were rendering directly to the backbuffer.

  • So we should make a texture importer/processor or something like that in the XNA content pipeline?

    I'm really interested in this stuff, these last two posts were really nice.

  • I have come up with a rather interesting way to do alpha blending. I have a 2D deformable landscape which is altered using 'brushes'. There is one image with the format 32bpp ARGB.

    I use the content pipeline to turn it into the following format: 32bpp premultiplied A8R8G8B8 + iA8 (that confuses even me).

    So the RGB component is premultiplied. The iA is inverted alpha which is ((1 << 7) * (1 - alpha / 255f)). In other words, it's a Q7 float stored in a byte. A is just stored as-is.

    At runtime I basically for each pixel:

    pixel.RGB = brush.RGB + ((pixel.RGB * a) >> 7));

    pixel.A = (byte)(Math.Min(255, brush.A + pixel.A));

    With all these optimizations in place I am getting about 12.3 megapixels per second (I also have a subtraction brush which is adding a few cycles for each pixel) on a Core 2 Duo Extreme 3GHz 64bit OS. Obviously some cycles are going toward the texture chunking (how I wish drawing 2048x2048 textures was a compatible reality).

    Because I am using Q-number arithmetic the alpha isn't 100% precise, but definitely nothing perceptible.

    Premultiplied alpha r0x0rz.

  • I would make one correction:

    "Premultiplied alpha blending is associative, so any number of blending operations can be grouped and reordered without affecting the final result."

    The term "reordered" is deceiving, because blending is still not commutative.

    Here are some thoughts I collected on premultiplied alpha (some plagiarized from here), because after using it for two years, I'm on the fence about how much I really need it. I've run into a few drawbacks recently, such as sRGB issues, which I thought I'd share with your readers.

    PROS:

    -----

       * You can combine blending and additive in one draw call.

       * Reflections on top of translucent surfaces, like water, glass etc. can be done in a single pass (glass can't be done in a single pass with traditional blending because the low alpha kills the reflections as well).

       * Linear combinations work correctly, which happens to fix DXT1 fringing (since transparency is only supported on black texels in DXT1, linear blending to black causes dark fringes in separated space).

       * Associativity works, so you can render some transparent stuff to a separate target, ex: "bc" in a(bc)d. But you can also achieve this by rendering only "bc" with premultiplied alpha, and the rest traditionally.

    CONS:

    -----

       * Most people (artists) don't understand it, so mistakes are easy to make:

             o Forgetting that everything coming into a shader, including textures, vertex colors, and color parameters must be premultiplied (unless you wanna do extra work in the shader).

             o Changing alpha only and forgetting to fix the premultiply.

             o It's too easy to cause slightly additive effects, knowingly or not, which can wreak havoc on bloom.

       * sRGB doesn't work correctly on PS3 with premultiplied alpha because conversion on the alpha channel is not supported (at least on the pixel shader output), requiring hacking and a performance penalty in the shader.

       * Some extra math required in shaders:

             o can't just change translucency by manipulating alpha, must multiply the change into the color as well.

             o to handle fog correctly, etc.

       * DXT5 artifacts become more apparent at the fringes because the compression on the color and alpha channels is different.

       * You lose some precision and information in the color channels. Only matters if you ever need to do something with the color other than straight blending (for example if you're trying to color correct red, 50% translucent red is not pure red anymore).

  • PS: You should do an article on sRGB as well, as that's another consideration that has a huge visual impact, yet it is virtually unknown and everybody gets it wrong, even going as far as the hardware manufacturers themselves.

Page 1 of 1 (6 items)
Leave a Comment
  • Please add 1 and 5 and type the answer here:
  • Post