Thanks Andrew for pointing this out -

Get it here:

Please note that XNA Game Studio Express requires Visual C# Express Edition to be installed. Both Visual C# Express and XNA Game Studio Express can be installed side-by-side with other Visual Studio 2005 SKUs (the normal caveats about installing beta software apply, of course).

Now, go write some games!