Automatic XNB serialization in XNA Game Studio 3.1

Automatic XNB serialization in XNA Game Studio 3.1

Rate This
  • Comments 38

One of the new features in Game Studio 3.1 is automatic serialization for .xnb files. This is a feature I have wanted ever since we first designed the Content Pipeline, so it makes me very happy that we finally found time to implement it!

 

Short Version

You know those ContentTypeWriter and ContentTypeReader thingamyhickies? You don't need them any more! Delete them, and the Content Pipeline will automatically serialize your data using reflection.

Note: you can still write a ContentTypeWriter and ContentTypeReader by hand if you want. You just don't have to any more.

 

Ever So Slightly Longer Version

If the Content Pipeline encounters a type that has no matching ContentTypeWriter, it will dynamically create one for you. This works in a similar way to the IntermediateSerializer, but it reads and writes binary .xnb files instead of XML.

  • By default it will serialize all the public fields and properties of your type (assuming they are not read-only).

  • If you want to skip a public member, mark it with a [ContentSerializerIgnore] attribute.

  • If you want to include a private, protected, or internal member, mark it with a [ContentSerializer] attribute.

  • If you want to serialize a data structure that contains cyclic references, mark these with [ContentSerializer(SharedResource = true)].

 

Example

The trick to successfully using the Content Pipeline is to understand which code runs at build time versus runtime. Even though the serialization is now automatic, you can still get in a muddle if you have things like cyclic references where your game tries to use a type that is defined inside itself while building itself!

Here's an example of using the new serializer:

Create a new Windows Game project. Let's call this MyGame.

Right-click on the Solution node, Add / New Project, and choose the Windows Game Library template (not Content Pipeline Extension Library, because we want to use this at runtime as well as build time). Call it MyDataTypes.

Add this class to the MyDataTypes project:

    public class CatData
    {
        public string Name;
        public float Weight;
        public int Lives;
    }

We're going to use this type to build some custom content, so we need to reference it during the Content Pipeline build process. Right-click on the Content project that is nested inside MyGame, choose Add Reference, and select MyDataTypes from the Projects tab.

Now we can add this file (let's call it cats.xml) to our Content project:

    <?xml version="1.0" encoding="utf-8" ?>
    <XnaContent>
      <Asset Type="MyDataTypes.CatData[]">
        <Item>
          <Name>Rhys</Name>
          <Weight>17</Weight>
          <Lives>9</Lives>
        </Item>
        <Item>
          <Name>Boo</Name>
          <Weight>11</Weight>
          <Lives>5</Lives>
        </Item>
      </Asset>
    </XnaContent>

Hit F5, and the content will build. Look in the bin directory, and you will see that it created a cats.xnb file. If you build in Release configuration, this file will be compressed. With the above example, my 326 byte XML file becomes 270 bytes in .xnb format, but the compression ratio will improve as your files get bigger.

Before we can load this data into our game, we must reference the MyDataTypes project directly from MyGame as well from its Content sub-project, in order to use our custom type at runtime as well as build time. Once we've done that, we can load the custom content:

    CatData[] cats = Content.Load<CatData[]>("cats");

If you create a copy of this project for Xbox 360, you will notice that although the Xbox version of MyGame references the Xbox version of MyDataTypes, its Content project still uses the Windows version of MyDataTypes, even though it is building content for Xbox. This is an important point to understand. Because our custom type is used both at build time (on Windows) and at runtime (on Xbox), we must provide both Windows and Xbox versions of this type.

 

Type Translations

Remember how some types are not the same at build time versus runtime? For instance a processor might output a Texture2DContent, but when you load this into your game it becomes a Texture2D.

The .xnb serializer understands such situations, as long as you give it a little help. For instance if I used this type in MyGame:

    public class Cat
    {
        public string Name;
        public Texture2D Texture;
    }

I could declare a corresponding build time type in a Content Pipeline extension project:

    [ContentSerializerRuntimeType("MyGame.Cat, MyGame")]
    public class CatContent
    {
        public string Name;
        public Texture2DContent Texture;
    }

Note how both types have basically the same fields, and in the same order, but the runtime Cat class uses Texture2D where the build time version has Texture2DContent. Also note how the build time CatContent class is decorated with a ContentSerializerRuntimeType attribute. This allows me to use CatContent objects in my Content Pipeline code, but load the resulting .xnb file as type Cat when I call ContentManager.Load.

 

Performance

Every silver lining has a cloud, right?

The automatic .xnb serialization mechanism uses reflection. Reflection is slow, and causes a lot of boxing, which can result in a lot of garbage collections.

Fortunately, this is not as bad in practice as it sounds on paper. Many custom data types are small, or at least the custom part of them tends to contain just a few larger objects of types that have built-in ContentTypeWriter implementations (textures, vertex buffers, arrays or dictionaries of primitive types, XNA Framework math types, etc). In such cases the performance overhead will be low. When I converted a bunch of existing Content Pipeline samples to use the new serializer, it did not significantly affect their load times.

But if you have a collection holding many thousands of custom types, you may see poor load performance. In such cases, you can provide a ContentTypeWriter and ContentTypeReader for just the specific types that are slowing you down. The new system is easier to use, but it can be more efficient to do things the old manual way.

  • Phil: no promises (the only way to be 100% sure is to try this both ways and time it with your particular data) but in general I would expect XNB serialization to be much faster than XML.

  • something weird happens ! it tells me that it can't find the XML file .. i didn't try serializing before to tell u the truth.. but this really is bugging me..

  • Is it possible to use serialize Dictionary objects? And if so, how should they be defined the  XML file?

  • What happens when you serialize a sub-class of another class? My game objects all have common attributes which are within the base class. Future classes sub-class and provide additional functionality.

    Does it automatically grab the super classes serializable attributes?

  • > Does it automatically grab the super classes serializable attributes?

    Of course!

    More technically, it calls into the ContentTypeWriter for the base class, which might be either manually implemented or automatically generated (the base class writer doesn't have to be created the same way as that for the derived type).

  • Another question related to XNB serialization for game objects in "level files."

    Is there any support for different "versions" of the class being serialized? My current solution is to mark public attributes as [ContentSerializer(Optional = true)]

    when I add a new attribute to the class.

    If I need version support would I have to keep track of a version value for the "game object" and then write my own reader/writer using that version number?

  • > Is there any support for different "versions" of the class being serialized

    The only built in support for .xnb versioning is the TypeVersion property on ContentTypeWriter and ContentTypeReader, which deliberately prevents loading if the version has changed.

    I guess you could build some more flexible mechanism of your own, but I wouldn't really recommend this. The general thinking is that .xnb files are compiled data, so they don't really need to be versioned on the fly. It's usually easier to just recompile them from the source content (which obviously does need to have good versioning and backward compatibility) any time the version changes, which keeps the runtime loading code nice and simple.

  • What does source content with good versioning and backward compatibility look like? Are there examples of this on the creators club website?

    My current routine is to convert the XML content for my game objects to the latest version using Find/Replace. I haven't figured out a good system to make it automated.

  • The FBX file format is an example of something with good backward compatibility.

    In general, designing a format and writing a loader that supports multiple versions is a lot of work. The Content Pipeline does nothing to help with this, but also does nothing to get in your way: this is entirely down to what code you put in your importer, so any standard computer sciencey format versioning techniques can be used.

    Depending on the amount of data you are dealing with and how many people you have creating it, in many cases you may find it easier to just manually fix up any old files rather than doing the work to automate this.

  • If I have a hierarchy of types like this:

       public class Animal

       {

           public string Name;

       }

       public class Cat : Animal

       {

           public int Age;

       }

    How do I create a corresponding content type (or types) decorated with the ContentSerializerRuntimeType attribute, so that I can load the Cat type at runtime without writing a custom ContentTypeReader/Writer for it?

    I tried this:

       [ContentSerializerRuntimeType("Cat, MyAssembly")]

       public class CatContent

       {

           public string Name;

           public int Age;

       }

    Which works during the build phase.  However when I attempt to call Content.Load<Cat>("ACat") I receive a ContentLoadException.

    Microsoft.Xna.Framework.Content.ContentLoadException was unhandled

     Message="Error loading \"ACat\". Cannot find ContentTypeReader for Animal."

     Source="Microsoft.Xna.Framework"

     StackTrace:

          at Microsoft.Xna.Framework.Content.ContentTypeReaderManager.GetTypeReader(Type targetType, ContentReader contentReader)

          at Microsoft.Xna.Framework.Content.ContentTypeReaderManager.GetTypeReader(Type targetType)

          at Microsoft.Xna.Framework.Content.ReflectiveReader`1.Initialize(ContentTypeReaderManager manager)

          at Microsoft.Xna.Framework.Content.ContentTypeReaderManager.ReadTypeManifest(Int32 typeCount, ContentReader contentReader)

          at Microsoft.Xna.Framework.Content.ContentReader.ReadHeader()

          at Microsoft.Xna.Framework.Content.ContentReader.ReadAsset[T]()

          at Microsoft.Xna.Framework.Content.ContentManager.ReadAsset[T](String assetName, Action`1 recordDisposableObject)

          at Microsoft.Xna.Framework.Content.ContentManager.Load[T](String assetName)

          at Nexus.Client.Client.Initialize() in C:\Source\NexusC#\Client\Client.cs:line 97

          at Microsoft.Xna.Framework.Game.Run()

          at Nexus.Client.Program.Main(String[] args) in C:\Source\NexusC#\Client\Program.cs:line 14

     InnerException:

  • chris:

    a) it's generally better to use the creators.xna.com forums for this kind of support question - blog comments aren't really the best place for posting code snippets!

    b) With automatic serialization, your design time inheritance hierarchy needs to match the runtime one, so you need an AnimalContent base class for use at design time. If your design time and runtime types are structurally different, you cannot use automatic serialization, and will have to write an old-style ContentTypeWriter/Reader pair for them.

  • We have an article with some additional tipps to consider with XNB serialization on our blog http://blog.brightside-games.com/?p=65 so maybe that can answer some of your questions here. best Thomas

  • Hi Shawn,

    I hope you don't mind me posting a) to an old topic and b) with such a (probably) stupid question... But I have been going around in circles for days trying to find the best way around this problem, and I am at wit's end!

    Would it help if I described my problem as being pernicious? ;)

    I have implemented a content importer (without using a ContentTypeReader/Writer, since I am using XNA 3.1) which reads a XML file and creates a List of Sprites.

    (This Sprite object is in a (referenced) second project, as per the guidance on one of your previous posts).

    However, I now want to derive classes (Enemies, Scenery Tiles, Players etc) from this Sprite object, as I don't want the various specialized methods within the base Sprite class.

    I naively expected to be able to downcast from a (say) Sprite object to a Player, but I am unable to do so, as the importer creates instances of the base class (Sprite).

    (I should at this point say that I thought I understood down-casting, but I have always used it from an object that had previously been up-cast)

    How should I approach this design issue?

    Anyway, please forgive me for asking such a newbie question!

    Cheers,

  • Mike: you can only cast a Sprite object to a Player if the object in question is in fact a Player instance. How do you originally create these objects? You will need to make sure you create them all using the correct types for each. The serialization mechanism will preserve whatever types you give it so my guess is you are not creating the right types in the first place.

    btw. I would recommend the forums on creators.xna.com if you have further questions - those tend to work better than blog comments for this kind of support discussion!

  • Thank you for this post.It was realy helpful!

Page 2 of 3 (38 items) 123
Leave a Comment
  • Please add 6 and 8 and type the answer here:
  • Post