SpriteBatch billboards in a 3D world

SpriteBatch billboards in a 3D world

  • Comments 34

A while ago I wrote about how to use SpriteBatch with a custom vertex shader, but didn't go into detail about how to set up matrices for drawing sprites in 3D.

The fundamentals are simple:

  • SpriteBatch generates vertex data containing Vector3 positions (plus texture coordinates and tint colors)
  • Each sprite is parallel to the XY plane
  • XY coordinates are computed from the position, rotation, origin and scale SpriteBatch.Draw parameters
  • The layerDepth SpriteBatch.Draw parameter is stored directly into the Z coordinate
  • If you use a Draw overload that does not specify a layerDepth, it defaults to zero

You can move sprites to any 3D position by applying the appropriate transforms to this vertex data. If you have a detailed understanding of shader coordinate systems, you now know how to position text in a 3D world or use SpriteBatch for 3D particle systems. If not, keep reading...

 

The identity transform

Let's make this concrete by working through a real example. Start by downloading the Billboards sample (which I chose because it has a fully movable camera, so we can easily view our 3D sprites from different directions). Add a Sprite Font asset to the content project, and call it "font". Add these fields to the BillboardGame class:

    SpriteBatch spriteBatch;
    SpriteFont spriteFont;
    BasicEffect basicEffect;

Initialize them in the LoadContent method:

    spriteBatch = new SpriteBatch(GraphicsDevice);
    spriteFont = Content.Load<SpriteFont>("font");

    basicEffect = new BasicEffect(GraphicsDevice)
    {
        TextureEnabled = true,
        VertexColorEnabled = true,
    };

At the end of the Draw method (right before the call to base.Draw) we will render some text using SpriteBatch plus BasicEffect:

    spriteBatch.Begin(0, null, null, null, null, basicEffect);
    spriteBatch.DrawString(spriteFont, "hello, world!", Vector2.Zero, Color.White);
    spriteBatch.End();

But when we run the program, no text appears! What gives?

  • SpriteBatch is generating vertices for a text string with (0,0) as its top left corner, sized so one unit equals one texel of the font texture
  • In the SpriteBatch coordinate system, X increases to the right and Y increases downward
  • Since we didn't set the BasicEffect transform matrices, it defaults to identity and passes these positions through unchanged
  • The output is interpreted in homogenous projection space, which ranges from -1 to 1
  • So our SpriteBatch vertex positions are way off the edge of the screen!
  • In homogenous projection space, X increases to the right while Y increases upward
  • This is the opposite way up to the SpriteBatch coordinate system, so our sprites are inside out and thus get backface culled

We can make the text visible by changing our drawing code to turn off backface culling and scale the text down to a fraction its original size:

    spriteBatch.Begin(0, null, null, null, RasterizerState.CullNone, basicEffect);
    spriteBatch.DrawString(spriteFont, "hello, world!", Vector2.Zero, Color.White, 0, Vector2.Zero, 0.005f, 0, 0);
    spriteBatch.End();

Now the message is visible, but ugly, stretched and upside down. We could fix this by carefully adjusting our scale factor to match the viewport size and aspect ratio, but that is a pain to get right and not my idea of fun. A better option is to change the BasicEffect transform matrices so the BasicEffect vertex shader will automatically apply the necessary coordinate transform.

 

The default SpriteBatch transform

To make BasicEffect emulate the default SpriteBatch behavior, we need an orthographic projection matrix. This should match the viewport size, and must invert the Y axis to convert SpriteBatch coordinates (where Y increases downward) to homogenous projection space (where Y increases upward):

    Viewport viewport = GraphicsDevice.Viewport;

    basicEffect.Projection = Matrix.CreateOrthographicOffCenter(0, viewport.Width, viewport.Height, 0, 0, 1);

    spriteBatch.Begin(0, null, null, null, null, basicEffect);
    spriteBatch.DrawString(spriteFont, "hello, world!", Vector2.Zero, Color.White);
    spriteBatch.End();

Now we can draw sprites the right way up and with no unwanted stretching, but everything appears blurry. This is because we have not correctly accounted for the texel centering offset, so our texture is unexpectedly filtered. We can fix this by adding a half pixel offset before the orthographic projection:

    Viewport viewport = GraphicsDevice.Viewport;

    basicEffect.Projection = Matrix.CreateTranslation(-0.5f, -0.5f, 0) * 
                             Matrix.CreateOrthographicOffCenter(0, viewport.Width, viewport.Height, 0, 0, 1);

    spriteBatch.Begin(0, null, null, null, null, basicEffect);
    spriteBatch.DrawString(spriteFont, "hello, world!", Vector2.Zero, Color.White);
    spriteBatch.End();

Tada! Even though we are using BasicEffect, everything now looks exactly the same as with the default SpriteBatch vertex shader. A lot of effort to end up exactly back where we started :-)

But wait, there's more...

 

Drawing sprites in 3D

We can set the BasicEffect projection matrices to anything we like. Try this version:

    Vector3 textPosition = new Vector3(0, 45, 0);

    basicEffect.World = Matrix.CreateScale(1, -1, 1) * Matrix.CreateTranslation(textPosition);
    basicEffect.View = view;
    basicEffect.Projection = projection;

    const string message = "hello, world!";
    Vector2 textOrigin = spriteFont.MeasureString(message) / 2;
    const float textSize = 0.25f;

    spriteBatch.Begin(0, null, null, DepthStencilState.DepthRead, RasterizerState.CullNone, basicEffect);
    spriteBatch.DrawString(spriteFont, message, Vector2.Zero, Color.White, 0, textOrigin, textSize, 0, 0);
    spriteBatch.End();

Notes:

  • We use the world matrix to invert the Y axis (so text will appear the right way up) and to translate the text to our desired position
  • We reuse the same view and projection matrices the sample has already computed, so the text appears in the same 3D scene as the rest of the terrain
  • As we move the camera around, we can view the text from different angles
  • Because text is alpha blended, we use DepthStencilState.DepthRead and draw it after all opaque geometry
  • We specify RasterizerState.CullNone so the text will be visible from both sides

 

Drawing billboard sprites

Sweet, we have text in 3D!  But it is fixed in place, with a static location and orientation. If the camera moves to view the text side on, it can no longer be read. If we were displaying something like a floating label over the head of a character, we'd probably want it to rotate and always face the camera. This is easily achieved using Matrix.CreateConstrainedBillboard:

    Vector3 textPosition = new Vector3(0, 45, 0);

    basicEffect.World = Matrix.CreateConstrainedBillboard(textPosition, textPosition - cameraFront, Vector3.Down, null, null);
    basicEffect.View = view;
    basicEffect.Projection = projection;

    const string message = "hello, world!";
    Vector2 textOrigin = spriteFont.MeasureString(message) / 2;
    const float textSize = 0.25f;

    spriteBatch.Begin(0, null, null, DepthStencilState.DepthRead, RasterizerState.CullNone, basicEffect);
    spriteBatch.DrawString(spriteFont, message, Vector2.Zero, Color.White, 0, textOrigin, textSize, 0, 0);
    spriteBatch.End();

 

Making billboards efficient

Fear not, the end is in sight...

What if we are drawing not just one piece of text, but a particle system containing hundreds or thousands of sprites? We could use the same code shown above, drawing each particle as a separate billboard. But because we are creating a separate billboard matrix for each sprite, which is set onto the BasicEffect, which is then passed to SpriteBatch.Begin, we would have to use a separate SpriteBatch Begin/End block for every single particle!  It'll work, but this will not be efficient.

We want to draw all the particles as a single batch, which means we cannot afford to change BasicEffect properties from one sprite to the next. We can still use BasicEffect for the projection matrix, but apply the view matrix transform on the CPU, then pass the resulting view space position to SpriteBatch.Draw, including its Z value as the SpriteBatch layerDepth:

    Matrix invertY = Matrix.CreateScale(1, -1, 1);

    basicEffect.World = invertY;
    basicEffect.View = Matrix.Identity;
    basicEffect.Projection = projection;

    spriteBatch.Begin(0, null, null, DepthStencilState.DepthRead, RasterizerState.CullNone, basicEffect);

    for each billboard sprite
    {
        Vector3 textPosition = new Vector3(0, 45, 0);

        Vector3 viewSpaceTextPosition = Vector3.Transform(textPosition, view * invertY);

        const string message = "hello, world!";
        Vector2 textOrigin = spriteFont.MeasureString(message) / 2;
        const float textSize = 0.25f;

        spriteBatch.DrawString(spriteFont, message, new Vector2(viewSpaceTextPosition.X, viewSpaceTextPosition.Y), Color.White, 0, textOrigin, textSize, 0, viewSpaceTextPosition.Z);
    }

    spriteBatch.End(); 

This produces the same result as the previous Matrix.CreateConstrainedBillboard example, but will be more efficient if we have many sprites to display.

Hopefully you now understand how any 2D particle system can be extended to draw in 3D, and how SpriteBatch can be a good way to draw 3D particles on Windows Phone.

  • "Great method for optimization. I'm curious, how would you set up a similar optimization for a sprite to billboard around a different axis, say, the Z axis?"

    I would also be very interested in this!

    I would also would like to know how to modify the last example to make the billboards NOT rotate (as in one of your earlier examples).

  • If the billboards don't rotate but are fixed to some other axis, then the need for that last optimization does not occur in the first place, as you can just draw everything using a single batch as shown in the first couple of examples.

  • True, but I want to draw them into a 3d world where they would scale like the billboards would, just not rotate. Which is exactly what one of your earlier examples did, unfortunately that was the unoptimized version (where you would need a spritebatch for each sprite you want to draw this way).

  • You mean perspective scaling so distant things get smaller?  That's no problem: just set a projection matrix and it happens automatically.  SpriteBatch by default, without any crazy tricks, can position quads anywhere you like in 3D space, then render them using any transform you care to apply.  The only constraint is that all these quads are facing the same direction (whatever direction that may be).  Crazy tricks are only required if you want each sprite to face a DIFFERENT direction, as is the case when doing 'face toward the camera' billboarding.

  • That sounds like what I'm looking for! Would you happen to have a link to an article of something?

    What I'm looking for is something that allows me to let SpriteBatch Draw 2D sprites on 3D positions, which I combination with a 3D camera get smaller/bigger depending on the distance. Maybe this article wasn't exactly what I'm looking for, but the itnernet is such a forest when looking for things like this.

  • That is exactly what the "Drawing sprites in 3D" part of this article describes, before it gets started on the billboard stuff.

  • I'm sorry if I sound stupid or anything and I appreciate your fats responses!

    Ok, I get that part, then back to my original question: Is there a way to do it WITHOUT making a spritebatch for each sprite I want to draw this way?

    If not: is making a spritebatch for each sprite really that inefficient?

  • > Is there a way to do it WITHOUT making a spritebatch for each sprite I want to draw this way?

    If all these sprites are on the same plane, yes.

    Otherwise, no.

  • > Is there a way to do it WITHOUT making a spritebatch for each sprite I want to draw this way?

    If all these sprites are on the same plane, yes.

    Otherwise, no.

  • Thank you for your time, I think I got it now!

    If you would see the Z as Z-depth in my case, I would need a SpriteBatch for each Z I have.

    As an illustration, I could make on SpriteBatch full of trees at Z = -10, but would need another SpriteBatch (Begin/End) to render say some grass at Z = -5. Is this correct?

  • If all you are doing is offseting the depth without rotation, you can specify a different layerDepth parameter for each draw call.

    You only need a different batch if you want different orientation, so the sprites are not all on parallel planes.

  • Great article Shawn!

    > You only need a different batch if you want different orientation, so the sprites are not all on parallel planes.

    Darn, I was hoping I would be able to position sprites at different 3D positions in the 3D world all with different 3D orientations, but it sounds like all of their orientations would need to be the same.

    Is it possible to modify your efficient billboard approach to have it allow each sprite to have a unique roll rotation (i.e. 2D rotation)? Or is this not possible without using a separate batch for each 2D rotation value (i.e 0 - 360 degrees)?  Thanks!

  • The important thing to understand is that SpriteBatch is just generating x/y/z vertex positions, which are then fed into whatever matrix transforms you specify. Once you understand that, you can figure out pretty much any application from first principles.

    SpriteBatch has complex logic for computing the x/y part of the vertex position (based on the position, scale, rotation, and origin specified for each Draw call), but the vertex z is just fixed to the value of the layerDepth parameter. Thus, all sprites are parallel and all perpendicular to the z axis.

    When you feed these vertices into a transform matrix, you can scale, rotate, project, whatever you like, but you can't change the fact that all the input sprites are parallel to each other. If you rotate in this transform, you could redirect so they are perpendicular to something other than the z axis, so it all depends on whether the output you want is achievable by a matrix transform over a set of input positions that SpriteBatch is capable of generating.

  • > Is it possible to modify your efficient billboard approach to have it allow each sprite to have a unique roll rotation (i.e. 2D rotation)?

    Bah, I was thinking that we would have to use a rotation matrix to rotate the sprites, but I just remembered that you can pass a float rotation value into the SpriteBatch's Draw() call, and this works for rotating the billboards shown in your example :)

    >Darn, I was hoping I would be able to position sprites at different 3D positions in the 3D world all with different 3D orientations, but it sounds like all of their orientations would need to be the same.

    Yay, I got this one figured out too by modifying your method in the "Drawing sprites in 3D" section. The trick is to use SpriteSortMode.Immediate in the Begin() call, and then just adjust the basicEffect.World matrix before drawing each sprite so that the sprite is oriented in the desired 3D orientation. I'm not sure how efficient this is, but it's nice to know at least that it can be done; so in a single SpriteBatch.Begin() call we can draw all of the sprites at different 3D positions with different 3D orientations! :)  If you can think of a more efficient way to do this please let me know.  Thanks!

  • SpriteSortMode.Immediate basically means "don't batch anything".  It's roughly equivalent to just wrapping every Draw call in its own Begin/End block.

    That's sometimes necessary, but defeats the entire point of batching, so will be terrible for performance if you are drawing more than a small number of sprites this way.

Page 2 of 3 (34 items) 123
Leave a Comment
  • Please add 8 and 2 and type the answer here:
  • Post