Automatic XNB serialization in XNA Game Studio 3.1

re: Automatic XNB serialization in XNA Game Studio 3.1

wmfwlr — Thu, 26 Mar 2009 07:25:21 GMT

This will come in very handy, for sure. I could have used it a month ago. I guess now I'll be one of those old guys who looks back and says "I remember when I had to write my own ContentTypeWriter. Kids nowadays have it easy".

re: Automatic XNB serialization in XNA Game Studio 3.1

CatalinZima — Thu, 26 Mar 2009 09:54:34 GMT

Ok, I was waiting for this post :) It's good to be finally be rid of writing those writers/readers.

Now, a question:

For the last example, the one with the Texture2DContent: what is the corresponding input XML for this example? Can the Texture2DContent be simply written in the XML (how?), or does it have to be initialized inside a ConentProcessor (because this is where you have access to the context)?

re: Automatic XNB serialization in XNA Game Studio 3.1

barkers crest — Thu, 26 Mar 2009 16:26:49 GMT

Shawn, with this new feature is it possible to serialize to xnb during runtime?

re: Automatic XNB serialization in XNA Game Studio 3.1

ShawnHargreaves — Thu, 26 Mar 2009 19:20:23 GMT

barkers crest: no. Writing .xnb files is done as part of the Content Pipeline build process, same as before.

re: Automatic XNB serialization in XNA Game Studio 3.1

ShawnHargreaves — Thu, 26 Mar 2009 19:23:51 GMT

Catalin: in theory you could embed the texture data directly into XML (TextureContent can be serialized by IntermediateSerializer) but I'm not sure how useful that would be!

More likely you would want to just store the texture filename in your XML, then go off and read that file in during the build process, which would require some processor code.

Here's an example from one of our unit tests of what a TextureContent looks like in IntermediateSerializer XML:

</Pixels>

</Mipmap>

</Pixels>

</Mipmap>

</Mipmaps>

</Asset>

</XnaContent>

re: Automatic XNB serialization in XNA Game Studio 3.1

CatalinZima — Fri, 27 Mar 2009 12:59:49 GMT

Ok, so I still need a Processor to solve this in an elegant (and resource-cheap) way, since textures will be shared between different assets (which means embedding them isn't too good).

re: Automatic XNB serialization in XNA Game Studio 3.1

ShawnHargreaves — Fri, 27 Mar 2009 20:21:25 GMT

I would most likely do this in a processor, yes. The auto serialization doesn't remove the need to write processor code if you want to process data or trigger other actions (like going off and building a texture) at runtime.

There are a couple of other options, though:

- you could just embed a string asset name in in the XML, then call ContentManager.Load on that name at runtime after you load the .xnb

- or you could embed an ExternalReference instance, in which case the .xnb deserializer will automatically follow that reference and load the asset it points to in place of the original reference

These options only work if the referenced asset is already being built through some other mechanism, though, so you know what name the reference should point to. They would be an option if you have a bunch of textures that are already added to your Content project, and can just put the name of these textures in this other XML asset, but unlike using a custom processor, they will not automatically go off and build that texture for you if the XML references a texture that is not otherwise part of your Content project.

re: Automatic XNB serialization in XNA Game Studio 3.1

CatalinZima — Thu, 02 Apr 2009 23:51:14 GMT

And unfortunately, I'd have no errors during build telling me that some textures are missing.

re: Automatic XNB serialization in XNA Game Studio 3.1

DCCreation — Wed, 13 May 2009 16:26:26 GMT

Just amazing !!!

Thank you really !

I work with XNA 3.0, and it couldn't auto serialize my XML objects, and I'm so happy to see this post !

It's a very good new. I earn a lot of time.

re: Automatic XNB serialization in XNA Game Studio 3.1

Kevin Smeltzer — Fri, 12 Jun 2009 19:26:55 GMT

I see that runtime serialization to .xnb will not be availble which makes sense because that is part of the build process.

Does that mean that this new fancy reflection based auto serialization will be useless for necessary things such as saving/loading game states, or can you use the framework to seriliaze to something other than .xnb at run time?

Thanks!

re: Automatic XNB serialization in XNA Game Studio 3.1

ShawnHargreaves — Fri, 12 Jun 2009 20:05:07 GMT

Kevin: this new serializer is part of the Content Pipeline, so the writing part is only available at build time.

For runtime serialization of things like save games, I would probably start with XmlSerializer, or if you need something more efficient, you could use BinaryWriter/BinaryReader to create a more compact binary format.

re: Automatic XNB serialization in XNA Game Studio 3.1

Kevin Gadd — Mon, 15 Jun 2009 09:28:17 GMT

So I've finally gotten around to trying out the content pipeline. Until now, I've been loading/saving all my game data using XML serialization, even on the 360, and it works fine - but I want to start deploying my content as packed XNB files for better load performance.

As a basic implementation, I wrote a raw importer to read in my XML files as byte[] and then a processor to convert those to my runtime data structures. This seems to work, but for some reason, the XNB files generated by this automatic reflection-based serializer are around 10 times larger than my XML files.

Is this behavior intended, or am I using the content pipeline incorrectly? Will the overhead remain fixed as my files grow larger? It's hard to tell whether or not it's fixed overhead from looking at the XNB files in a hex editor.

re: Automatic XNB serialization in XNA Game Studio 3.1

ShawnHargreaves — Mon, 15 Jun 2009 17:08:59 GMT

Kevin: this really depends on what your data looks like. XNB files are usually much smaller than XML (especially if you enable compression) but it's certainly possible to construct objects where this would not be the case.

I would recommend the forums on creators.xna.com if you want to follow up with more details, as they're somewhat nicer than these comment boxes for having a discussion :-)

re: Automatic XNB serialization in XNA Game Studio 3.1

Kevin Gadd — Tue, 16 Jun 2009 04:32:38 GMT

Luckily it turns out it was a bit of a false alarm :) The size went down significantly once I fiddled with some of the serialization attributes - there's still some overhead, but from testing it appears to be fixed - most of it comes from the fact that the ReflectiveWriter has to write out the fully qualified names of all the types it uses, which isn't the case in my source XML.

The actual binary data for the content seems to be quite compact, which is what I was hoping for.

re: Automatic XNB serialization in XNA Game Studio 3.1

Phil — Fri, 19 Jun 2009 22:50:45 GMT

I currently have several different (fairly large ~12KB) xml files that I copy to the output directory, and load when my game starts up. They seem to take a long time to load on the Xbox.

Would I expect to see a performance improvement by moving these files into content pipeline? (I would write a ContentImporter that deserializes from xml, and then uses this new automatic serialization functionality).

There would be some improvement from the smaller file sizes... but is the deserialization functionality generally faster than what I have now (the standard Xml deserialization)?

re: Automatic XNB serialization in XNA Game Studio 3.1

ShawnHargreaves — Fri, 19 Jun 2009 22:56:17 GMT

Phil: no promises (the only way to be 100% sure is to try this both ways and time it with your particular data) but in general I would expect XNB serialization to be much faster than XML.

re: Automatic XNB serialization in XNA Game Studio 3.1

mod — Fri, 10 Jul 2009 06:09:22 GMT

something weird happens ! it tells me that it can't find the XML file .. i didn't try serializing before to tell u the truth.. but this really is bugging me..

re: Automatic XNB serialization in XNA Game Studio 3.1

GrahamRanson — Mon, 13 Jul 2009 03:51:23 GMT

Is it possible to use serialize Dictionary objects? And if so, how should they be defined the XML file?

re: Automatic XNB serialization in XNA Game Studio 3.1

Paul Solt — Mon, 27 Jul 2009 10:11:33 GMT

What happens when you serialize a sub-class of another class? My game objects all have common attributes which are within the base class. Future classes sub-class and provide additional functionality.

Does it automatically grab the super classes serializable attributes?

re: Automatic XNB serialization in XNA Game Studio 3.1

ShawnHargreaves — Mon, 27 Jul 2009 18:49:53 GMT

> Does it automatically grab the super classes serializable attributes?

Of course!

More technically, it calls into the ContentTypeWriter for the base class, which might be either manually implemented or automatically generated (the base class writer doesn't have to be created the same way as that for the derived type).

re: Automatic XNB serialization in XNA Game Studio 3.1

Paul Solt — Tue, 11 Aug 2009 07:19:34 GMT

Another question related to XNB serialization for game objects in "level files."

Is there any support for different "versions" of the class being serialized? My current solution is to mark public attributes as [ContentSerializer(Optional = true)]

when I add a new attribute to the class.

If I need version support would I have to keep track of a version value for the "game object" and then write my own reader/writer using that version number?

re: Automatic XNB serialization in XNA Game Studio 3.1

ShawnHargreaves — Tue, 11 Aug 2009 20:51:33 GMT

> Is there any support for different "versions" of the class being serialized

The only built in support for .xnb versioning is the TypeVersion property on ContentTypeWriter and ContentTypeReader, which deliberately prevents loading if the version has changed.

I guess you could build some more flexible mechanism of your own, but I wouldn't really recommend this. The general thinking is that .xnb files are compiled data, so they don't really need to be versioned on the fly. It's usually easier to just recompile them from the source content (which obviously does need to have good versioning and backward compatibility) any time the version changes, which keeps the runtime loading code nice and simple.

re: Automatic XNB serialization in XNA Game Studio 3.1

Paul Solt — Wed, 12 Aug 2009 03:05:28 GMT

What does source content with good versioning and backward compatibility look like? Are there examples of this on the creators club website?

My current routine is to convert the XML content for my game objects to the latest version using Find/Replace. I haven't figured out a good system to make it automated.

re: Automatic XNB serialization in XNA Game Studio 3.1

ShawnHargreaves — Wed, 12 Aug 2009 03:10:43 GMT

The FBX file format is an example of something with good backward compatibility.

In general, designing a format and writing a loader that supports multiple versions is a lot of work. The Content Pipeline does nothing to help with this, but also does nothing to get in your way: this is entirely down to what code you put in your importer, so any standard computer sciencey format versioning techniques can be used.

Depending on the amount of data you are dealing with and how many people you have creating it, in many cases you may find it easier to just manually fix up any old files rather than doing the work to automate this.

re: Automatic XNB serialization in XNA Game Studio 3.1

Chris — Thu, 27 Aug 2009 00:56:18 GMT

If I have a hierarchy of types like this:

public class Animal

{

public string Name;

}

public class Cat : Animal

{

public int Age;

}

How do I create a corresponding content type (or types) decorated with the ContentSerializerRuntimeType attribute, so that I can load the Cat type at runtime without writing a custom ContentTypeReader/Writer for it?

I tried this:

[ContentSerializerRuntimeType("Cat, MyAssembly")]

public class CatContent

{

public string Name;

public int Age;

}

Which works during the build phase. However when I attempt to call Content.Load<Cat>("ACat") I receive a ContentLoadException.

Microsoft.Xna.Framework.Content.ContentLoadException was unhandled

Message="Error loading \"ACat\". Cannot find ContentTypeReader for Animal."

Source="Microsoft.Xna.Framework"

StackTrace:

at Microsoft.Xna.Framework.Content.ContentTypeReaderManager.GetTypeReader(Type targetType, ContentReader contentReader)

at Microsoft.Xna.Framework.Content.ContentTypeReaderManager.GetTypeReader(Type targetType)

at Microsoft.Xna.Framework.Content.ReflectiveReader`1.Initialize(ContentTypeReaderManager manager)

at Microsoft.Xna.Framework.Content.ContentTypeReaderManager.ReadTypeManifest(Int32 typeCount, ContentReader contentReader)

at Microsoft.Xna.Framework.Content.ContentReader.ReadHeader()

at Microsoft.Xna.Framework.Content.ContentReader.ReadAsset[T]()

at Microsoft.Xna.Framework.Content.ContentManager.ReadAsset[T](String assetName, Action`1 recordDisposableObject)

at Microsoft.Xna.Framework.Content.ContentManager.Load[T](String assetName)

at Nexus.Client.Client.Initialize() in C:\Source\NexusC#\Client\Client.cs:line 97

at Microsoft.Xna.Framework.Game.Run()

at Nexus.Client.Program.Main(String[] args) in C:\Source\NexusC#\Client\Program.cs:line 14

InnerException:

re: Automatic XNB serialization in XNA Game Studio 3.1

ShawnHargreaves — Thu, 27 Aug 2009 18:16:31 GMT

chris:

a) it's generally better to use the creators.xna.com forums for this kind of support question - blog comments aren't really the best place for posting code snippets!

b) With automatic serialization, your design time inheritance hierarchy needs to match the runtime one, so you need an AnimalContent base class for use at design time. If your design time and runtime types are structurally different, you cannot use automatic serialization, and will have to write an old-style ContentTypeWriter/Reader pair for them.

re: Automatic XNB serialization in XNA Game Studio 3.1

Thomas — Wed, 02 Sep 2009 13:39:12 GMT

We have an article with some additional tipps to consider with XNB serialization on our blog http://blog.brightside-games.com/?p=65 so maybe that can answer some of your questions here. best Thomas

re: Automatic XNB serialization in XNA Game Studio 3.1

Mike — Tue, 13 Oct 2009 12:41:08 GMT

Hi Shawn,

I hope you don't mind me posting a) to an old topic and b) with such a (probably) stupid question... But I have been going around in circles for days trying to find the best way around this problem, and I am at wit's end!

Would it help if I described my problem as being pernicious? ;)

I have implemented a content importer (without using a ContentTypeReader/Writer, since I am using XNA 3.1) which reads a XML file and creates a List of Sprites.

(This Sprite object is in a (referenced) second project, as per the guidance on one of your previous posts).

However, I now want to derive classes (Enemies, Scenery Tiles, Players etc) from this Sprite object, as I don't want the various specialized methods within the base Sprite class.

I naively expected to be able to downcast from a (say) Sprite object to a Player, but I am unable to do so, as the importer creates instances of the base class (Sprite).

(I should at this point say that I thought I understood down-casting, but I have always used it from an object that had previously been up-cast)

How should I approach this design issue?

Anyway, please forgive me for asking such a newbie question!

Cheers,

re: Automatic XNB serialization in XNA Game Studio 3.1

ShawnHargreaves — Tue, 13 Oct 2009 15:56:55 GMT

Mike: you can only cast a Sprite object to a Player if the object in question is in fact a Player instance. How do you originally create these objects? You will need to make sure you create them all using the correct types for each. The serialization mechanism will preserve whatever types you give it so my guess is you are not creating the right types in the first place.

btw. I would recommend the forums on creators.xna.com if you have further questions - those tend to work better than blog comments for this kind of support discussion!