Image codecs in XNA Game Studio 4.0

Image codecs in XNA Game Studio 4.0

  • Comments 32

Quick history lesson:

Once upon a time, we were designing XNA 1.0, and decided that instead of just exposing managed wrappers around the old D3DX functions for loading textures and models, it would be better to create a new thing called the Content Pipeline.

But when we came to release our 1.0 Beta, the Content Pipeline was not quite finished. Hmm. Doesn't seem like people are going to have much fun with a supposedly easy to use graphics API that has no built-in way to load textures! We needed a quick fix, so we slapped a hasty wrapper around the D3DX texture loader, and thus Texture2D.FromFile was born.

FromFile is only supported on Windows, because D3DX was not available to us on Xbox. We considered doing the extra work to port this to Xbox, but it seemed like a waste of effort to duplicate functionality that is also available through the Content Pipeline. We also considered removing FromFile from our final Windows version, but that seemed like an unnecessary breakage for our beta customers.

Game Studio 4.0 takes cross platform consistency more seriously, so this only-on-Windows situation was no longer acceptable. Plus, Windows Phone adds scenarios involving web services and picture library integration, where runtime image loading and saving are important. But the D3DX image code is large, full of obscure features, and would be expensive to port to all our target platforms.

We decided to replace our D3DX image loader with a simpler new API. The new methods are:

  • Supported consistently on Windows, Xbox 360, and Windows Phone
  • Support PNG, JPEG, and GIF formats
  • Provide options for scaling and cropping
  • Designed for dealing with pictures (screenshots, photo library, web services)
  • Not optimized for loading static game textures (you should use the Content Pipeline for that)
  • Less fully featured than the old D3DX loader (no mipmapping, DXT compression, or color key)

The new load method has two overloads:

    static Texture2D FromStream(GraphicsDevice GraphicsDevice, Stream stream);
static Texture2D FromStream(GraphicsDevice GraphicsDevice, Stream stream, int width, int height, bool crop);

Because this reads from a Stream rather than a filename, it can be used with many data sources: FileStream, MemoryStream, NetworkStream, TitleContainer.OpenStream, etc. To read from a file on disk, use it like so:

    using (Stream stream = File.OpenRead("cat.png"))
{
texture = Texture2D.FromStream(graphicsDevice, stream);
}

There are two save methods:

    void SaveAsPng(Stream stream, int width, int height);
void SaveAsJpeg(Stream stream, int width, int height);

Which are typically used like so:

    using (Stream stream = File.OpenWrite("cat.png"))
{
texture.SaveAsPng(stream, texture.Width, texture.Height);
}

The first FromStream overload does minimal scaling. As long as the source fits within the maximum GPU texture size (2048 for Reach, or 4096 for HiDef) it will be returned exactly as-is. For instance this photo of my dog Kess is loaded into a 192x288 SurfaceFormat.Color texture:

source

If the source is larger than the supported maximum, it will be scaled down as little as possible to make it fit, while preserving aspect ratio (so width and height are always shrunk the same amount).

The second FromStream overload offers more control over how the image is scaled, which is useful for photo viewer applications, thumbnail displays, etc. When crop is set to false, the image is scaled to fit entirely within the specified size, while preserving aspect ratio. For instance if I load the above photo with width=256, height=256, crop=false, I get:

256x256-False

The height has been scaled to 256, while the width has been scaled down to 170 (to preserve aspect). If my source image was wider than it was tall, the width would come back exactly 256, while the height would be smaller.

If I load the same photo with width=256, height=256, crop=true, I get:

256x256-True

The image has been scaled to entirely fill the specified size, and cropped as necessary to preserve aspect ratio, so the leaves from the top of the original photo are no longer visible.

Another way to think of this:

  • Crop=false means "scale the longer axis to exactly the specified size, leaving the shorter axis smaller than specified"
  • Crop=true means "scale the shorter axis to exactly the specified size, cropping the longer axis to make it fit"

Various places where we used to expose a Texture2D property now return a Stream instead. For instance, the Picture.GetTexture method (which returned a Texture2D) is now Picture.GetImage (which returns a Stream). Likewise for album art and gamer profile pictures. The returned stream contains a JPEG encoded image.

This change was neccessary to make these APIs compatible with Silverlight. By decoupling image sources from the XNA Texture2D class, it is now possible to use our media API both from an XNA game (which can use FromStream to convert the stream into an XNA Texture2D) and also from Silverlight (which can load that same stream into a Silverlight BitmapImage).

  • I like it alot. Looks very useful.

    On 360 saving pictures was very annoying and hard before, and I couldn't figure out how to do jpeg compression.

  • I'm wondering, after reading all these posts - are there any new features in 4.0? Something you can do that you couldn't before. It seems like it is all API refactoring and minor tweaks.

    Anyways, my 2 biggest wishes for the content pipeline are:

    1. Debugging support.

    2. Deployment with the game for user-generated content.

    Don't know if that will ever happen...

  • Does the content pipeline still use the D3DX image routines?

    Because for the content pipeline they are not ideal:

    * Initializing a new D3D device every time another file is processed is slow.

    *In addition there are problems with device creation to process an image during desktop switchs(eg requests for admin privlages).

    * Plus wierd problems running the content pipeline in a profiler (eg SlimTune, PIX, NProf etc). ie xna throws exceptions when creating a device to process a texture. [yes I reported this to connect... a few years ago]

  • Is this method also available on the Xbox ?

    Will i be able to host a web service on a server that will serve images and access it from the xbox ?

  • > I'm wondering, after reading all these posts - are there any new features in 4.0? Something you can do that you couldn't before. It seems like it is all API refactoring and minor tweaks.

    The new features are summarized here: http://blogs.msdn.com/shawnhar/archive/2010/03/09/in-which-hints-become-facts-xna-game-studio-4-0.aspx

    By far the biggest new thing is support for Windows Phone.

  • > Is this method also available on the Xbox ?

    From the article above:

    - Supported consistently on Windows, Xbox 360, and Windows Phone

    > Will i be able to host a web service on a server that will serve images and access it from the xbox ?

    No. Xbox does not support any network access other than the Xbox LIVE NetworkSession API.

  • On a slightly different subject about textures: Is the content pipeline gonna support the ATI2N format?

    I'm making do with Swizzled-DXT5 for now, but it would be nicer to have access to the best normal map standard.

  • > Is the content pipeline gonna support the ATI2N format?

    This is not just about the Content Pipeline: XNA does not support any such non-standard texture formats at all.

    In some distant future if/when we add a third profile level over the top of Reach and HiDef, that would most likely expose the new DX10 block compressed formats, but for now, no.

  • > Will i be able to host a web service on a server that will serve images and access it from the xbox ?

    No. Xbox does not support any network access other than the Xbox LIVE NetworkSession API.

    How is it possible to use the new method along with a NetworkStream object?

    Also, is this method Synchroneous or Asynchroneous?

  • > How is it possible to use the new method along with a NetworkStream object?

    On Xbox, it is not, since there is no NetworkStream class on Xbox.

    The point is that image codecs are decoupled from any particular I/O implementation, so you can use the same image load/save API with whatever stream implementations are available on whatever platform you are working on.

  • Why the heck there is dog on a picture instead of cat...

    It's not prerecorded for posting, so who are You, who impersonate as Shawn?!

  • Curious, behind the scenes are you using WIC on windows?

    WIC is mind bogglingly fast.

  • Simple question: Development must be performed on Windows Vista or Windows 7 (can't even download the xna 4.0 on my xp system). Games generated in XNA 4.0, are they vista/Windows 7 only or can they also be executed on XP? I'd check myself, but I'm one of those people that didn't upgrade to Vista and have been putting off Windows 7 for a while longer. Probably after XNA 4.0 is in full release I'll make my move.

  • I'm using XNA 3.1 for a 3D level editor. The lack of any reasonable ability to load textures from outside of the content pipeline is a major deal-breaker. I hope you reconsider this.

  • >SaveAsPng, SaveAsJpeg

    I don't like this anti-OOP API. What happened to good old System.Drawing syntax for image saving?

    What about BMP/TGA?

    This should be texture.Save(stream, ImageFormat.Png)

Page 1 of 3 (32 items) 123
Leave a Comment
  • Please add 7 and 3 and type the answer here:
  • Post