Thanks Andrew for pointing this out -
Get it here:
Please note that XNA Game Studio Express requires Visual C# Express Edition to be installed. Both Visual C# Express and XNA Game Studio Express can be installed side-by-side with other Visual Studio 2005 SKUs (the normal caveats about installing beta software apply, of course).
Now, go write some games!