Somebody asked here on the forums if you can use VS 2003 to debug .NET 2.0 (whidbey) apps. Unfortunately, the answer is no.
VS 2003 can not debug .NET 2.0 apps.  It is a restriction in the underlying .NET debugging services (see below)
You can still use the .NET 2.0 SDK tools (such as Mdbg, DbgClr) to debug .NET 2.0 apps.  However, Visual Studio 2005 is bar far the best tool to be using for .NET 2.0 apps.

Backwards is easier than forwards:
Now VS 2005 can debug existing .NET 1.0, 1.1 apps. It's much easier to go backwards than forwards. In fact, there is technically nothing the CLR can do stop VS from doing this (not that we would want to) because in a worst case, VS 2005 could have both a VS2005 and VS2003 debug engine. However we did try to make this much easier by keeping ICorDebug relatively intact from 2003 to 2005.

Last I checked, the VS2005 project system only wants to build .NET 2.0 apps. (It will try to convert 2003 projects to 2005 projects).  (update:) See John Rivard's explanation for why VS only targets its matching CLR.

In general, debuggers can't do source level debugging on new compilers because the debuggers usually need intimate information that change with the new compilers. For the CLR, future CLRs can add radically new concepts that confuse older CLRs. For eg,  .NET 2.0 has generics and discontinuous methods, which would confuse a debugger targeting .Net 1.0.  Another more general example is that the new compilers can revise the PDB formats, and then the old debuggers are broken.  (Matt Pietrek elaborates on this further here)

At about this point, somebody always has to mention that you can take a native debugger from 15 years ago and attach it to an app compiled today. That's true, but you'll be debugging at the assembly level - and issue here is debugging source  level. That same 15 year old debugger could also debug a .NET 2.0 app ... at the assembly level.

See CreateDebuggingInterfaceFromVersion() for more on the technical details about the CLR Debugging services versioning.