.NET Compact Framework Team : Author: Chris To

Managed Code Performance on Xbox 360 for XNA: Part 2 - GC and Tools

NetCFTeam — Sat, 23 Dec 2006 00:14:00 GMT

...continuation of Part 1, it can be found here

Memory and Garbage Collection

One common concern for game developers is the garbage collector. By design, GCs trade off determinism for convenience. Luckily, keeping the GC predictable is fairly straightforward. Two variables to pay attention to are:

How long a GC is taking (aka. GC latency)
When and how often a GC happens

You’ll want to strike a balance between “how long” and “how often” to get smooth game play. Even if your GCs are few and far in between such that your average framerate is unaffected, a single GC that takes long enough will cause a perceivable skip. That’s pretty obvious… so how long you ask? We observed that you see a small skip at around 25-30ms of latency on top of the running frame rate. That is under the assumption that a GC is happening just about once per second. Which leads in to the second factor, lots of regular GCs will kill your average framerate with unnecessary overhead. One thought that occurs to people is: “Why not call GC.Collect() every frame so it’s deterministic!”. The wasted overhead you get for “over collecting” typically doesn’t make sense.

At 60fps each frame takes about 16.67ms (that’s 1000ms/60frames). Isolated benchmarks show that a GC of 100,000 live objects takes about 14ms (though that’s a relatively large number of objects compared to Rocket Commander by Benjamin Nitschke which has just under 50,000 live objects at peak). Would you really want to sacrifice 80% of your game forcing GCs (~14ms/17ms)? Don’t think so. One disclaimer, the referenced 14ms benchmark is “isolated” and doesn’t account for the cost of compaction and collection which are additional real world costs depending on heap fragmentation and general object churn.

So how does one control GC latency? Like NetCF for devices, the Xbox GC is non-generational. That means every collection is a full collection on the managed heap. Thus, we find that GC latency is approximately linear to the number of live objects… then add the cost of heap compaction on to that. Our benchmarks show that the difference between deep object hierarchies vs. shallow ones is negligible, so it’s mostly the number of objects that matter. Small objects also tend to be a somewhat cheaper to deal with than big objects.

The triggers that cause collection are unchanged from v2.0 as well. A collection is triggered for every 1MB allocated or if an allocation fails from an out-of-memory exception.

Games typically have lots of small objects that represent game state. The obvious optimization here is to reduce live object count. You can do that by defining those data structures as structs which are value types (to use more general terminology). Value types stay off the GC heap... of course that assumes that your structs don’t get boxed in to objects, which can often happen unknowingly in your code.

A concise blog posting about value type boxing can be found here: http://blogs.msdn.com/scottholden/archive/2005/01/27/362084.aspx. An obvious boxing is scenario is, casting a value type to an object to call an object method. An insidious boxing scenario is passing a value type to a method that takes an object type (eg. ArrayList.Add(object)). Generics help you avoid this latter scenario, but can also lead to other non-obvious boxing scenarios. You can use the XNA Remote Performance Monitor Tool discussed below to watch out for boxing. Read more about the generics implementation in NetCF here: http://blogs.msdn.com/romanbat/archive/2005/01/06/348114.aspx.

One more caveat on structs that was touched on in the floating point section: avoid passing and returning large value types by value, and instead use “ref” or “out”. I’ll emphasize again that passing and returning large value types like Matrix is particularly costly in NetCF. Of course, make sure to understand the semantics of ref/out here: http://msdn2.microsoft.com/en-us/library/8f1hz171.aspx

The second obvious optimization is to reduce the number of allocations over the lifetime of your game. You can do that by pooling data structures and reusing them rather than repeatedly “newing” up new instances. There are many benefits here ranging from saving on the cost of “new” to theoretically reducing the need to compact the heap because it isn’t churning all the time.

Combine both tactics together: recycle structs in constant size array pools while avoiding boxing and that should mitigate any GC perf worries! We had a reasonably complex game (well visually complex at least) do all of the above and it’s GC latency averaged 6ms so it was never an issue.

Tools

The Remote Performance Monitor from 2.0 SP1 is back with a slightly upgraded UI as the XNA Remote Performance Monitor for Xbox 360! It’s easy to set up and connect and will give you basic but valuable performance counter data. Leading in from the last section, the GC counters will probably be the most valuable. Notice the fields: “Garbage Collections”, “GC Compactions”, “Boxed Value Types”, “GC Latency Time”.

Watch the numbers tick and try to keep them from ticking up based on the tips above. Yep. That’s all.

Oh yeah, and make sure to measure and profile your code before you start optimizing... System.Diagnostics.Stopwatch. (You can't have a blog about perf without mentioning that)

Look forward to seeing games that push the limits of the system so we know where to optimize next!

Managed Code Performance on Xbox 360 for XNA: Part 1 - Intro and CPU

NetCFTeam — Fri, 22 Dec 2006 23:24:00 GMT

Introduction

Now that XNA Game Studio Express 1.0 is out, it’s time to start writing managed code for the Xbox 360. Performance is a popular topic that’s been on this blog and the Xbox is likely no exception. The bottom line for gaming performance is framerate. The de facto framerate standard these days seems to be 60fps. I emphasize that bottom line because it’s tempting to get in to technical minutiae when talking about performance, for example CPU cache or pipeline efficiency, but that’s not really the spirit of XNA Game Studio Express (with an emphasis on Express here). The goal of this blog posting is to provide some insight in to the NetCF team’s performance expectations for this release along with guidance on where you’ll get the most “bang” for your perf optimizing “buck”.

Graphics

XNA pretty much gives you full access to the powerful Xbox 360 graphics hardware which supports a superset of Shader Model 3.0. Efficient use of the rendering pipeline is critical to game performance and it’s often the right area to focus on first… assuming you’re doing anything graphically interesting. Besides… isn’t the visceral appeal of graphics the reason why everyone wants to program games! Though AI might fit in there somewhere too.

That said an in depth look at graphics is going to be outside the scope of this posting. A detailed discussion of how to maximize the XNA graphics performance is best left to the XNA team that built it. We just did the runtime!

CPU

So on to the CPU… since the Xbox 360 CLR is based on the .Net Compact Framework, much of the existing knowledge sitting around on the internet for NetCF applies just as well to the Xbox. Here are some examples:

.Net Compact Framework version 2.0 Performance and Working Set FAQ:

http://blogs.msdn.com/netcfteam/archive/2005/05/04/414820.aspx

Desktop perf advice often applies to NetCF as well:

Improving Managed Code Performance (J.D. Meier, Srinath Vasireddy, Ashish Babbar, Rico Mariani, and Alex Mackman):

http://msdn.microsoft.com/library/en-us/dnpag/html/scalenetchapt05.asp?frame=true

Taming the CLR: How to Write Real-Time Managed Code (via Rico Mariani’s blog)

http://blogs.msdn.com/ricom/archive/2006/08/22/713396.aspx

Measuring managed code quickly and easily: CodeTimers (via Vance Morrison’s blog)

http://blogs.msdn.com/vancem/archive/2006/09/21/765648.aspx

If you’re new to managed code performance, I highly recommend the “Improving Managed Code Performance” link. It covers a lot of fundamentals in a comprehensible way: efficient class design, boxing, exceptions, collections, etc. Pretty much everything.

Floating Point

Now that all said, both the Xbox 360 hardware and gaming scenarios are very different from what NetCF has been traditionally optimized for. The NetCF 2.0 JIT compiler for Windows CE does not support hardware floating point. In fact, hardware floating point isn’t supported in the ARMv4 architecture that current Windows Mobile devices target.

Thankfully, the Xbox 360 version of NetCF now does support hardware floating point. That work resulted in over a 10-fold improvement in our micro-benchmarks. So we think we’re satisfied with floating point perf for now. Nevertheless, it hasn’t been maximized in this release and that‘s an area of focus for the future. For example, the Xbox 360 also supports VMX128 (often referred to as AltiVec), but NetCF doesn’t take advantage of that hardware today.

Expect to get less floating point throughput than you would from identical managed code running on the desktop CLR on a good off the shelf PC. How much less depends on your code so it’s hard to provide a meaningful ratio. In most cases, other performance “gotchas” will probably hit you before floating point becomes a dominating factor, for example excessive virtual function calls or over-zealous object allocation.

If you do suspect that floating point is keeping your game from the magic 60fps number, make sure to try out the static “by ref” and “by out” methods provided by the XNA math APIs such as Vector4.Add(ref Vector4 , ref Vector4, out Vector4). The more convenient binary operators (like "+" in this case) pass their arguments by value. Using ref and out is a useful trade-off that can yield significant perf improvements by avoiding extra overhead associated with passing large structs by value.

If you’re implementing some sort of mathematically intensive model perhaps as part of a fancy particle system, make sure to think about what floating point operations can be offloaded to the GPU via shaders in case you haven’t looked in to that already. Shaders get the full throughput of the Xbox GPU that the pro game developers get. You’ll just need to trade-in C# for some HLSL.

Inlining

Manual inlining is a useful way to improve perf by removing unnecessary function call overhead. NetCF will handle some method inlining on your behalf, but it’s limited to very simple methods like basic getters/setters. Thus it’s worthwhile for developers to keep inlining in mind too. Methods that get called a lot (eg. big loops are a place to look) are good candidates to manually inline (assuming said method is reasonably sized). Check out the NetCF “Method Inlining” section on this link for more context: http://blogs.msdn.com/netcfteam/archive/2005/05/04/414820.aspx. Note that virtual methods never get inlined automatically.

Multiple Cores

The Xbox 360 has 6 hardware threads on 3 cores. Support for multiple hardware threads was also added to System.Threading so XNA developers can access 4 of the 6 hardware threads (2 are reserved) via Thread.SetProcessoryAffinity(). You will have to assign your threads manually to take advantage of the hardware threads as the Xbox kernel doesn’t do it automatically for you. Note that this API isn’t available on the desktop.

Of course the general concepts behind efficient multithreaded coding and synchronization apply.

Here’s a place to get started on the .Net threading APIs:

http://msdn2.microsoft.com/en-us/library/3e8s7xdd.aspx

This article is more of an editorial and admittedly more interesting read that also happens to address multi-CPU and hyper threaded systems like the Xbox (by Joe Duffy):

http://msdn.microsoft.com//msdnmag/issues/06/09/CLRInsideOut/default.aspx

Once you’ve got that stuff down, you should keep in mind that hardware threads are different from cores. Each core has two hyper threads that share the same cache. Putting two threads that access totally separate data on the same core can lead to unnecessary thrashing of the cache. XNA GSE doesn’t provide tools to really see what’s going on at the CPU level anyways, so if anything you can time your code and use some trial and error to determine what works best.

To be continued... the next part will cover GC and Memory Management, and Tools.

Updated... Part 2 is here.