I've given sample code for MDbg-based harnesses that launch an app and then print all loaded modules or print exceptions that occur.  In both cases, the harness launches the app. One reader asked how to modify them to attach to an existing app. It's actually very easy.
In either case, we need to get an MDbgProcess instance. In the launch case, we get it via CreateProcess (which ultimately calls ICorDebug::CreateProcess) with something like:
    MDbgProcess proc = debugger.CreateProcess(szProgramName, "", DebugModeFlag.Default, null);

To attach to an existing process, we need to switch this over to calling ICorDebug::DebugActiveProcess() on a given pid. That could look like this:
    int pid =  (... get pid of interest ...);
    MDbgProcess proc = debugger.Attach(pid);

I just tried out the change on the Load-Module harness and it worked on the first time.

Perhaps a related interesting question is how to figure out the pid to attach to given the name. See here for sample code to answer that question.

API differences between Launch and Attach:
Once you have the MDbgProcess or ICorDebugProcess, the rest of the API looks the same.  ICorDebug will actually fake up certain notifications (LoadModule, CreateThread, etc) by enumerating all modules, threads, etc that exist at the time of attach and then firing "fake" callbacks for them.   (The native debugging API does this too)
I have mixed feelings about this. On one hand, firing "fake" callbacks is lying, and lying causes grief in APIs. On the other hand, it sure can be convenient because it unifies both the Launch and Attach scenarios. After all, we only had to change a single line in the example to go from Launch to Attach.

Also, there is no managed "attach complete" callback. This is in contrast to the native API which does have a "loader breakpoint" (99.9% of the time, the first int 3). Thus there is not a good way in V2.0 to tell when you've stopped getting "fake" events and started getting "real" events again.

[Update: 11/19/05]  MDbg tries to help out here. Once you Go() after an Attach(), it will pump all the "fake" events and return a psuedo "AttachCompleteStopReason". This is very useful if you want a harness that attaches and does inspection instead of sniffs for debug events.