The XNA Framework Content Pipeline

So we've talked about XNA Game Studio Express, and just the other day Mitch posted all about the XNA Framework that XNA Game Studio Express uses. Let's talk about another piece of the XNA Game Studio Express product—namely, the XNA Framework Content Pipeline. As I noted in my Gamefest slides, the XNA Framework Content Pipeline is best described as:
 
"An extensible content processing framework managed in C# Express"
 
That may not tell you everything you need or want to know about this technology, though, so let's try and start from the beginning.
 
Content is a large part of games today, and getting content into games isn't easy—in fact, it's frustrating. You face so many problems, whether you’re looking for an exporter or the right tool to support that exporter. Then you have to find some tools to process the content, and use those tools to create data for your game. You also need to worry about loading the content and doing all the work necessary to get the content displaying correctly in your game. I summed that up pretty quickly, but it’s a lot of work. In fact, large studios have teams of people who work only on putting this infrastructure in place.
 
With the XNA Framework Content Pipeline, the process is different—and improved. It is simpler to use; it's highly extensible and customizable to what you're doing; and it gives you choices when putting your game together, both in the tools you use to create content and in the engine you use. We know it's not very exciting to build game content infrastructure when what you really want to do is focus on your game. The XNA Framework Content Pipeline lets you do just that: focus on developing your game!
 
One of the first things worth talking about is how your content is used in your game. With the XNA Framework Content Pipeline, your content is managed inside Visual C# Express. So just as you would add code files today, you'll add your content as well. This helps keep things together and lets you organize your entire game in one solution, with any number of projects you want. You'll be able to set actions for your content such as what Importer and Processor to use. (More on those in a bit!)
 
Many different components make up the XNA Framework Content Pipeline. For each component we discuss, I've called out its name. Becoming familiar with these terms will help you better comprehend the XNA Framework Content Pipeline.
 
When you first add your content, you need to pick an importer. An importer is responsible for taking data and normalizing it. This means you don't have to worry about, for example, what direction your content is facing or which way is up in your content creation tool. The importer takes the files you've saved or exported from your content creation tools and imports them into Visual C# Express. The reason for this will become clear as you read about the rest of the system, but I want to point out that we're more concerned about the data in the file. In other words, we’re not as tied to which content creation tool that data came from.
 
So what importers exist for V1 of XNA Game Studio Express? Take a look at this handy chart.

Most of these importers are probably familiar to you, but one worth discussing is Autodesk FBX. FBX is a 3D transport file format that allows you to move 3D scene data across many different content creation tools. Not only is FBX supported in most commercial 3D tools,  it's also supported in many shareware and freeware tools. Even outside of that, we tried hard to pick formats that multiple tools can consume and write.
 
After the importer has done its job, the data exists in our content DOM. The term DOM is used simply to represent a collection of classes or a schema (like an XML file). The data sits in the content DOM as strongly typed data. That means the data is represented in a well-known format, so, for example, a series of vertices or texture data looks the same no matter what file it came from. This point is important in order for the next step to work properly. This file can be written out to disk as XML for debugging, but it's "lossy," which means the importer only stored the data it cared about.
 
A processor is responsible for taking data from the content DOM and creating an object for you to use at run time. This object can be as simple as a model, or as complex as multiple processors for your game. The XNA Framework Content Pipeline includes a few processors that ship out of the box such as Model (for simple objects with textures), Texture2D (for sprites or to combine with a Model), and Effect (For handling your Model's materials). This means you no longer need to worry about, for example, vertex buffers or triangle strips, but you might still want more flexibility with your content, since your game’s world may not be built on simple Models alone.
 
Here's an example of what I'm talking about. If you're putting together a racing game, you'd probably have some content files that make up your track. In that track, you'd likely have all kinds of data such as start/end points, locations that define checkpoints, and a spline that defines your track's path. Instead of breaking that track down into several hundred models, you could simply write a processor to consume all this data and spit out a track object. Yes, your engine needs to understand what a track object is, but processors offer a high degree of flexibility and customizability for any kind of content you want to use.
 
Processors are also designed to be easy to write, and easy to share and reuse. Remember that processors are pulling data from the content DOM and how I said that it was important? That's because your processor will work regardless of the file format it came from. Because all that data was stored in the same format, you don't have to worry about customizing your processor for a .x or .tga file. Additionally, you get helper functions, which let you manipulate and generate new data from the content DOM. These functions mimic a lot of what you'll be familiar with if you've used D3DX. A simple example might be generating mipmaps (that is, creating smaller versions of a graphic).
 
Content building is also an important part of our system and is handled by the build coordinator. Because your content is in Visual C# Express, when you click Build, all your content will be built, saved to disk, and ready for you to use at run time. We also provide an incremental build for content. This means if you change a texture, we rebuild only those items that were used with that texture.
 
Finally, you need a way to get that data back in your game. This is a task for the Content Manager. This component is designed to load assets quickly without a lot of fuss. It handles all the little details such as ensuring that all the associated assets are loaded at the same time. While this post isn't meant to be more than an overview, here's some code that shows how simple it really is!

 

ContentManager myManager = new ContentManager(GameServices);

model = myManager.Load<Model>("ship");

Sometimes when describing something it helps to have something to look at, so I put together this chart to show how the pieces fit together:

Extensibility is also an important part of the XNA Content Pipeline story. All the components that we've talked about are designed so that you can extend them. We'll talk more about that in another post.
 
Keep in mind that the XNA Framework Content Pipeline is a large, new piece of technology. Though we have a lot of people working hard on it, it simply wasn't ready in time for the beta on Wednesday. So the XNA Framework Content Pipeline is not part of the beta that you can download on August 30th. You'll still be able to use content in a game—it will just be much more of a manual process. The Spacewar Starter Kit will also be available to show you how developers are working with files outside the XNA Framework Content Pipeline.  We're still looking into other means of getting this to you before the release, but there are no solid plans yet.
 
Finally, as an added bonus, I've included a short demo of the XNA Framework Content Pipeline in action! I showed this demo at Gamefest and had it recorded especially for the readers of our team blog. Again, this demo doesn't dive into the code too much—there are plenty of articles on that—but it still shows that it works and gives you a feel of the overall experience.

XNA Framework Content Pipeline Demo (Streaming)
XNA Framework Content Pipeline Demo (Downloadable)
 
I hope this sheds some light on the other side of XNA Game Studio Express. We'll dive into it more technically as time goes on, but I wanted to start out with an overview since there are a lot of new concepts here to grasp.
 
Michael Klucher
Program Manager - XNA Game Studio