The Problem and Solutions

One thing that can be really tricky to track and and figure out are dynamic assemblies.  A common situation where they may be a problem is if your process isn't using much memory, yet you are running out of memory.  This is usually do to fragmentation and dynamic assemblies are a good cause of such fragmentation.  They are created in certain situations by .NET and they don't go away until the AppDomain, in which they were created in, is unloaded.  Take a look at this blog for more info on unloading an Assembly:
http://blogs.msdn.com/suzcook/archive/2003/07/08/57211.aspx

Some common issues we see around dynamic assemblies along with their solutions are:
  1. PRB: Cannot unload assemblies that you create and load by using script in XSLT
    • Note that for this issue, if you have just a few XSL templates and use them over and over again, consider caching them and re-using them.
  2. 872800 FIX: Web service clients regenerate serialization assemblies every time that the application runs
  3. Memory usage is high when you create several XmlSerializer objects in ASP.NET

There is also some information and another sample at:
http://blogs.msdn.com/tess/archive/2006/02/15/532804.aspx

Another alternative that you can use, if you know you are going to be creating dynamic assemblies and can't control them using these KB articles, you can consider creating a class that creates an AppDomain of it's own and unloads it periodically.  You can do various things to control when to unload the AppDomain, this sample just creates 100 dynamic assemblies as an example and to prove that they go away.  Such a class would look like:

using System;
using System.Reflection;
using System.Reflection.Emit;

namespace AppDomainCreation
{
    /// <summary>
    /// Summary description for AppDom.
    /// </summary>
    public class AppDom : MarshalByRefObject
    {
        public AppDom()
        {
        }

        public void DoIt()
        {
            int i = 0;
            try 
            {    
                for (i = 0; i < 100; i++)
                {
                    AppDomain myCurrentDomain = AppDomain.CurrentDomain;
                    AssemblyName myAssemblyName = new AssemblyName();
                    myAssemblyName.Name = "TempAssembly";

                    // Define a dynamic assembly in the current app domain.
                    AssemblyBuilder myAssemblyBuilder = 
myCurrentDomain.DefineDynamicAssembly(
myAssemblyName, AssemblyBuilderAccess.Run); // Define a dynamic module in this assembly. ModuleBuilder myModuleBuilder =
myAssemblyBuilder.DefineDynamicModule("TempModule"); } } catch (Exception ex) { throw ex; } } } }

And code to use it:

private void button1_Click(object sender, System.EventArgs e)
{
    myDom = AppDomain.CreateDomain("MyDom");
    AppDom my1 = (AppDom) myDom.CreateInstanceFromAndUnwrap(
        Assembly.GetAssembly(typeof(AppDom)).Location,
        typeof(AppDom).FullName);
    my1.DoIt();
}

private void button2_Click(object sender, System.EventArgs e)
{
    AppDomain.Unload(myDom);
}

Troubleshooting

As to how can you tell is dynamic assemblies are your problem, the sos.dll that ships with the debugger has some very useful commands to use.  The first is !dumpdomain -stat which gives a count of the number of assemblies in each domain:

0:000> !clr10\sos.dumpdomain -stat
    Domain 	Num Assemblies 	Size Assemblies 	Name
0x793ec4f8 	             1 	      2,088,960 	System Domain
0x793ed928 	            23 	     10,990,592 	Shared Domain
0x0014c698 	             2 	      2,473,984 	DefaultDomain
0x00185a10 	            19 	      7,796,736 	/LM/W3SVC/2/Root-1-127338058389375000
0x001b41f8 	         3,035 	     26,482,688 	/LM/W3SVC/4/Root-2-127338058403593750
0x0022d7b0 	            54 	      9,381,376 	/LM/W3SVC/3/Root-3-127338058446093750
0x2718e398 	            26 	      8,027,136 	/LM/W3SVC/9/Root-4-127338085958750000
0x2dcf5c68 	            13 	      7,430,144 	/LM/W3SVC/8/Root/test-5-127338086849687500
0x2dd2dfb8 	            16 	      7,433,216 	/LM/W3SVC/8/Root/test2-6-127338103281875000
0x2dd1d970 	            15 	      7,632,896 	/LM/W3SVC/8/Root/test3-7-127338159155781250
0x2dd47ea8 	            57 	      9,062,400 	/LM/W3SVC/7/Root-8-127338660348281250
0x2dd27148 	            11 	      7,290,880 	/LM/W3SVC/1/Root/test4-9-127338766413437500
Total 12 Domains, Total Size 106,091,008

Then you can run !dumpdynamicassemblies which will print out the individual assemblies:

0:000> !clr10\sos.dumpdynamicassemblies
Domain: ....
-------------------
Domain: 
-------------------
Domain: DefaultDomain
-------------------
Domain: /LM/W3SVC/2/Root-1-127338058389375000
-------------------
Assembly: 0x1ee1f0 [n-l5hxaj] Dynamic Module: 0x1e62c8
    loaded at: 0x26c01000 Size: 0x1000((null))
Assembly: 0x2943d470 [4k0trn5d] Dynamic Module: 0x27208e08
    loaded at: 0x2a201000 Size: 0x1000((null))
Domain: /LM/W3SVC/4/Root-2-127338058403593750
-------------------
Assembly: 0x271f0348 [ndbs9yfe] Dynamic Module: 0x13b6a8
    loaded at: 0x0 Size: 0x0((null))
Assembly: 0x271f0348 [ndbs9yfe] Dynamic Module: 0x193098
    loaded at: 0x0 Size: 0x0((null))
Assembly: 0x27236540 [ndbs9yfe] Dynamic Module: 0x272374c8
    loaded at: 0x29d71000 Size: 0x1400((null))
Assembly: 0x27169dd0 [3outmwsh] Dynamic Module: 0x1a14f0
    loaded at: 0x29da1000 Size: 0x1c00((null))
Assembly: 0x27249500 [fegr-cqw] Dynamic Module: 0x1a1908
    loaded at: 0x0 Size: 0x0((null))
Assembly: 0x27249500 [fegr-cqw] Dynamic Module: 0x1a1af8
    loaded at: 0x0 Size: 0x0((null))
Assembly: 0x27249438 [fegr-cqw] Dynamic Module: 0x293d43e8
    loaded at: 0x29e21000 Size: 0x1400((null))
Assembly: 0x27249370 [r4dtnazh] Dynamic Module: 0x293b3da0
    loaded at: 0x29e41000 Size: 0x1c00((null))
Assembly: 0x293d59e8 [tlq2vuw-] Dynamic Module: 0x293d6158
    loaded at: 0x0 Size: 0x0((null))
Assembly: 0x293d59e8 [tlq2vuw-] Dynamic Module: 0x293d6ab8
    loaded at: 0x0 Size: 0x0((null))

To see what each one if from, take the module address of the dynamic assembly and run !dumpmodule -mt <address>:

0:000> !clr10\sos.dumpmodule -mt 0x272374c8 
Name Unknown Module
dwFlags 0x00200080
Attribute PEFile 
Assembly 0x27236540
LoaderHeap* 0x27237548
TypeDefToMethodTableMap* 0x29d80010
TypeRefToMethodTableMap* 0x29d80020
MethodDefToDescMap* 0x29d80058
FieldDefToDescMap* 0x29d80090
MemberRefToDescMap* 0x29d800a0
FileReferencesMap* 0x29d800dc
AssemblyReferencesMap* 0x29d800e0
MetaData starts at 0x29d73258 (0x794 bytes)

Types defined in this module

        MT    TypeDef Name
------------------------------------------------------------------------------
0x29bc777c 0x02000003 Microsoft.Xslt.CompiledScripts.JScript.ScriptClass_4
0x29bc789c 0x02000004 Microsoft.Xslt.CompiledScripts.JScript.ScriptClass_1

Types referenced in this module

        MT    TypeRef Name
------------------------------------------------------------------------------
0x29c43c18 0x01000007 Microsoft.JScript.INeedEngine
0x79b7c364 0x01000008 System.Object

With .NET 2.0, we don't have these commands, but you can still get the data out.  To start with, run !dumpdomain and see if you get a lot of assemblies, ie:

0:000> !dumpdomain
...
Assembly: 1a66bb00 [xr3zkp-t, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]
ClassLoader: 1a66bb88
SecurityDescriptor: 1a68aa50
  Module Name
1ed839e8 xr3zkp-t, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null

Assembly: 1a6b3c30 [settdg8d, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]
ClassLoader: 1a6b3538
SecurityDescriptor: 1a6b3ba8
  Module Name
1ed84204 settdg8d, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null

Assembly: 1a6b4670 [isuhtubr, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]
ClassLoader: 1a6b3f78
SecurityDescriptor: 1a6b45e8
  Module Name
1ed84804 isuhtubr, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null

Assembly: 1a6b4b20 [d-9cnxi4, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]
ClassLoader: 1a6b4ba8
SecurityDescriptor: 1a6b4148
  Module Name
1ed84e24 d-9cnxi4, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null
...

And then run the same !dumpmodule -mt <address> on the Module address listed in question:

0:000> !dumpmodule -mt 1ed88a34 
Name: c84csakg, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null
Attributes: PEFile 
Assembly: 1a673740
LoaderHeap: 00000000
TypeDefToMethodTableMap: 1f0ef4e0
TypeRefToMethodTableMap: 1f0ef4f8
MethodDefToDescMap: 1f0ef568
FieldDefToDescMap: 1f0ef5e4
MemberRefToDescMap: 1f0ef64c
FileReferencesMap: 1f0ef7d0
AssemblyReferencesMap: 1f0ef7d4
MetaData start address: 1eeb2274 (7308 bytes)

Types defined in this module

      MT    TypeDef Name
------------------------------------------------------------------------------
1ed88f9c 0x02000002 Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationWriterbeBizTalkRequest
1ed88e94 0x02000006 Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializerContract

Types referenced in this module

      MT    TypeRef Name
------------------------------------------------------------------------------
69e5b5b8 0x01000001 System.Xml.Serialization.XmlSerializationWriter
69e5bbac 0x01000004 System.Xml.Serialization.XmlSerializerImplementation
0eccf4dc 0x01000005 MPI.Common.Webservices.beBizTalkRequest
022395bc 0x01000006 MPI.Common.Entities.beClientContext
0f444974 0x01000007 MPI.Insurance.DriverLicence.beIWSDriverLicenceFetchRequestPayload
0f444afc 0x01000008 MPI.Insurance.DriverLicence.beIWSDriverLicenceFetch
790fea70 0x0100000a System.Collections.Hashtable
79101058 0x0100000b System.Type
790f9c18 0x01000011 System.Object
69e359d0 0x01000015 System.Xml.XmlConvert