SpriteBatch billboards in a 3D world

SpriteBatch billboards in a 3D world

  • Comments 34

A while ago I wrote about how to use SpriteBatch with a custom vertex shader, but didn't go into detail about how to set up matrices for drawing sprites in 3D.

The fundamentals are simple:

  • SpriteBatch generates vertex data containing Vector3 positions (plus texture coordinates and tint colors)
  • Each sprite is parallel to the XY plane
  • XY coordinates are computed from the position, rotation, origin and scale SpriteBatch.Draw parameters
  • The layerDepth SpriteBatch.Draw parameter is stored directly into the Z coordinate
  • If you use a Draw overload that does not specify a layerDepth, it defaults to zero

You can move sprites to any 3D position by applying the appropriate transforms to this vertex data. If you have a detailed understanding of shader coordinate systems, you now know how to position text in a 3D world or use SpriteBatch for 3D particle systems. If not, keep reading...

 

The identity transform

Let's make this concrete by working through a real example. Start by downloading the Billboards sample (which I chose because it has a fully movable camera, so we can easily view our 3D sprites from different directions). Add a Sprite Font asset to the content project, and call it "font". Add these fields to the BillboardGame class:

    SpriteBatch spriteBatch;
    SpriteFont spriteFont;
    BasicEffect basicEffect;

Initialize them in the LoadContent method:

    spriteBatch = new SpriteBatch(GraphicsDevice);
    spriteFont = Content.Load<SpriteFont>("font");

    basicEffect = new BasicEffect(GraphicsDevice)
    {
        TextureEnabled = true,
        VertexColorEnabled = true,
    };

At the end of the Draw method (right before the call to base.Draw) we will render some text using SpriteBatch plus BasicEffect:

    spriteBatch.Begin(0, null, null, null, null, basicEffect);
    spriteBatch.DrawString(spriteFont, "hello, world!", Vector2.Zero, Color.White);
    spriteBatch.End();

But when we run the program, no text appears! What gives?

  • SpriteBatch is generating vertices for a text string with (0,0) as its top left corner, sized so one unit equals one texel of the font texture
  • In the SpriteBatch coordinate system, X increases to the right and Y increases downward
  • Since we didn't set the BasicEffect transform matrices, it defaults to identity and passes these positions through unchanged
  • The output is interpreted in homogenous projection space, which ranges from -1 to 1
  • So our SpriteBatch vertex positions are way off the edge of the screen!
  • In homogenous projection space, X increases to the right while Y increases upward
  • This is the opposite way up to the SpriteBatch coordinate system, so our sprites are inside out and thus get backface culled

We can make the text visible by changing our drawing code to turn off backface culling and scale the text down to a fraction its original size:

    spriteBatch.Begin(0, null, null, null, RasterizerState.CullNone, basicEffect);
    spriteBatch.DrawString(spriteFont, "hello, world!", Vector2.Zero, Color.White, 0, Vector2.Zero, 0.005f, 0, 0);
    spriteBatch.End();

Now the message is visible, but ugly, stretched and upside down. We could fix this by carefully adjusting our scale factor to match the viewport size and aspect ratio, but that is a pain to get right and not my idea of fun. A better option is to change the BasicEffect transform matrices so the BasicEffect vertex shader will automatically apply the necessary coordinate transform.

 

The default SpriteBatch transform

To make BasicEffect emulate the default SpriteBatch behavior, we need an orthographic projection matrix. This should match the viewport size, and must invert the Y axis to convert SpriteBatch coordinates (where Y increases downward) to homogenous projection space (where Y increases upward):

    Viewport viewport = GraphicsDevice.Viewport;

    basicEffect.Projection = Matrix.CreateOrthographicOffCenter(0, viewport.Width, viewport.Height, 0, 0, 1);

    spriteBatch.Begin(0, null, null, null, null, basicEffect);
    spriteBatch.DrawString(spriteFont, "hello, world!", Vector2.Zero, Color.White);
    spriteBatch.End();

Now we can draw sprites the right way up and with no unwanted stretching, but everything appears blurry. This is because we have not correctly accounted for the texel centering offset, so our texture is unexpectedly filtered. We can fix this by adding a half pixel offset before the orthographic projection:

    Viewport viewport = GraphicsDevice.Viewport;

    basicEffect.Projection = Matrix.CreateTranslation(-0.5f, -0.5f, 0) * 
                             Matrix.CreateOrthographicOffCenter(0, viewport.Width, viewport.Height, 0, 0, 1);

    spriteBatch.Begin(0, null, null, null, null, basicEffect);
    spriteBatch.DrawString(spriteFont, "hello, world!", Vector2.Zero, Color.White);
    spriteBatch.End();

Tada! Even though we are using BasicEffect, everything now looks exactly the same as with the default SpriteBatch vertex shader. A lot of effort to end up exactly back where we started :-)

But wait, there's more...

 

Drawing sprites in 3D

We can set the BasicEffect projection matrices to anything we like. Try this version:

    Vector3 textPosition = new Vector3(0, 45, 0);

    basicEffect.World = Matrix.CreateScale(1, -1, 1) * Matrix.CreateTranslation(textPosition);
    basicEffect.View = view;
    basicEffect.Projection = projection;

    const string message = "hello, world!";
    Vector2 textOrigin = spriteFont.MeasureString(message) / 2;
    const float textSize = 0.25f;

    spriteBatch.Begin(0, null, null, DepthStencilState.DepthRead, RasterizerState.CullNone, basicEffect);
    spriteBatch.DrawString(spriteFont, message, Vector2.Zero, Color.White, 0, textOrigin, textSize, 0, 0);
    spriteBatch.End();

Notes:

  • We use the world matrix to invert the Y axis (so text will appear the right way up) and to translate the text to our desired position
  • We reuse the same view and projection matrices the sample has already computed, so the text appears in the same 3D scene as the rest of the terrain
  • As we move the camera around, we can view the text from different angles
  • Because text is alpha blended, we use DepthStencilState.DepthRead and draw it after all opaque geometry
  • We specify RasterizerState.CullNone so the text will be visible from both sides

 

Drawing billboard sprites

Sweet, we have text in 3D!  But it is fixed in place, with a static location and orientation. If the camera moves to view the text side on, it can no longer be read. If we were displaying something like a floating label over the head of a character, we'd probably want it to rotate and always face the camera. This is easily achieved using Matrix.CreateConstrainedBillboard:

    Vector3 textPosition = new Vector3(0, 45, 0);

    basicEffect.World = Matrix.CreateConstrainedBillboard(textPosition, textPosition - cameraFront, Vector3.Down, null, null);
    basicEffect.View = view;
    basicEffect.Projection = projection;

    const string message = "hello, world!";
    Vector2 textOrigin = spriteFont.MeasureString(message) / 2;
    const float textSize = 0.25f;

    spriteBatch.Begin(0, null, null, DepthStencilState.DepthRead, RasterizerState.CullNone, basicEffect);
    spriteBatch.DrawString(spriteFont, message, Vector2.Zero, Color.White, 0, textOrigin, textSize, 0, 0);
    spriteBatch.End();

 

Making billboards efficient

Fear not, the end is in sight...

What if we are drawing not just one piece of text, but a particle system containing hundreds or thousands of sprites? We could use the same code shown above, drawing each particle as a separate billboard. But because we are creating a separate billboard matrix for each sprite, which is set onto the BasicEffect, which is then passed to SpriteBatch.Begin, we would have to use a separate SpriteBatch Begin/End block for every single particle!  It'll work, but this will not be efficient.

We want to draw all the particles as a single batch, which means we cannot afford to change BasicEffect properties from one sprite to the next. We can still use BasicEffect for the projection matrix, but apply the view matrix transform on the CPU, then pass the resulting view space position to SpriteBatch.Draw, including its Z value as the SpriteBatch layerDepth:

    Matrix invertY = Matrix.CreateScale(1, -1, 1);

    basicEffect.World = invertY;
    basicEffect.View = Matrix.Identity;
    basicEffect.Projection = projection;

    spriteBatch.Begin(0, null, null, DepthStencilState.DepthRead, RasterizerState.CullNone, basicEffect);

    for each billboard sprite
    {
        Vector3 textPosition = new Vector3(0, 45, 0);

        Vector3 viewSpaceTextPosition = Vector3.Transform(textPosition, view * invertY);

        const string message = "hello, world!";
        Vector2 textOrigin = spriteFont.MeasureString(message) / 2;
        const float textSize = 0.25f;

        spriteBatch.DrawString(spriteFont, message, new Vector2(viewSpaceTextPosition.X, viewSpaceTextPosition.Y), Color.White, 0, textOrigin, textSize, 0, viewSpaceTextPosition.Z);
    }

    spriteBatch.End(); 

This produces the same result as the previous Matrix.CreateConstrainedBillboard example, but will be more efficient if we have many sprites to display.

Hopefully you now understand how any 2D particle system can be extended to draw in 3D, and how SpriteBatch can be a good way to draw 3D particles on Windows Phone.

  • Hi Shawn,

    First thank you for your work (both personnal and profesionnal).

    Your article makes me think about impostors, for example those used in assassin s creed

    where you can look around huge cities (i mean huuuge) seeing much detail

    without any slowndown of the framerate.I m not sure what technique ubisoft uses

    and how they put their impostors together so that the player can t notice the trick;

    maybe that impostors coding would be an interesting follow up to your articles ?

  • Thanks for the new interesting view Shawn, however some of the code samples above have been truncated.

    Is the code available for download, or are you planning to do a sample with the above at some point (assuming one is already available)

  • Maybe it's interesting to add to this article that you can also do the rendering with one set of vertices + world and camera, just use another vertexbufffer (instancebuffer) to get the positions to the gpu.

  • > some of the code samples above have been truncated.

    Which ones? They all look correct to me.

    Maybe your browser window is just too narrow to view the longer lines of code? If so you can either just expand your browser, or paste the code into Visual Studio.

  • > you can also do the rendering with one set of vertices + world and camera, just use another vertexbufffer (instancebuffer) to get the positions to the gpu

    You mean using instanced rendering? That's certainly a valid and useful graphics technique, but it's not really the same thing as what I'm talking about in this article. Instancing doesn't use SpriteBatch, and doesn't work on Windows Phone, for instance.

  • @Metalov: The Wolfire Blog has a great article on imposters : blog.wolfire.com/.../Imposters

    One thing I miss terribly in some of your articles Shawn are screenshots and images, in the past you put some for MotoGP and it was really helpful to understand some things.

    Anyway, kudos again for an helpful article ;)

  • Nice reading ...

    > some of the code samples above have been truncated.

    Yes, they are truncated but you can still do a full copy of the code.

    > We can fix this by adding a half pixel offset before the orthographic projection.

    My doubt: is the offet value the same (half a pixel) for viewport's resolution/sizes smaller than full screen?

  • > Yes, they are truncated but you can still do a full copy of the code.

    Or just get a wider monitor :-)

    > My doubt: is the offet value the same (half a pixel) for viewport's resolution/sizes smaller than full screen?

    Yep. This is in pixels, so it's always exactly half of one pixel regardless of the viewport size.

  • > Or just get a wider monitor :-)

    Lol

  • I have to agree with Remi Gillig, this article would be improved with some screen shots. Still a good article, thanks Shawn.

  • Hmm, for some reason my billboards are just white squares. I've tried with both text (DrawString) and a real texture (Draw), and tried changing the color as well, but it always comes out as just white squares. At least they're on the screen in my 3D scene where I need them to go, but I wonder what I'm missing.

  • Xoanan: did you set TextureEnabled on your BasicEffect?

  • Bah! RTFM strikes again! I was so sure that wouldn't need to be set since the SpriteBatch had the texture and would be taking care of it. Thanks Shawn. This article was exactly what I needed! Keep up the great work.

  • Great method for optimization. I'm curious, how would you set up a similar optimization for a sprite to billboard around a different axis, say, the Z axis?

  • Awesome article Shawn. I wonder how this approach compares performance-wise to drawing particles as textured quads.  With the removal of point sprites in XNA 4 my particle systems have become so much slower, so if this approach is faster than using textured quads that would be fantastic. I'll go ahead and implement it once I have time, but was curious if you did any initial performance comparisons.

Page 1 of 3 (34 items) 123
Leave a Comment
  • Please add 6 and 1 and type the answer here:
  • Post