Ok... I still have not gotten around to writing that uber blog entry/guide on the ins and outs of 64bit IIS6 and WOW64... so I wager I should take little bites here and there as motivation.
Writing big long technical articles that try to answer lots of things is hard for me. It is far easier if three users asked questions about various related aspects of WOW64 around the same time... because then I get motivated to answer all those questions at once, and I'll naturally collapse them into that one uber blog entry... but as fortune goes...
I am wanting to run an ASP application on Windows 2003 64 bit edition.Because some of the COM components are compiled on 32 bit machines i understand that i have to run IIS in WOW64 compatibilty mode. Does this mean i will have the same memory limitations per process of w3wp.exe (2GB) ? Do i get any benefit memory wise running IIS in this mode on windows 2003 64 bit edition? My main issue is that my Application Pool recycles too fast. I have it recycle at 1500mb because at a setting higher than this i get memory problems. Could i increase this to something like 3000mb on Windows 2003 64 bit and if so what would you reccommend the paging file size be on the disk and physical ram?
I'll answer your question in multiple parts because I do not see how COM components require WOW64 nor how it relates to Application Pool recycling and memory limitations.
I am not certain your situation requires running IIS6 in WOW64 compatibility mode.
So, I do not see a reason why you must run in WOW64 mode. There may be other issues involved, but not according to the rationale you gave.
I am a little confused by you saying that memory-based recycling at 1.5GB is reached "too-fast" yet your application gets memory problems if the value is set higher. It sounds like your application attempts to allocate large chunks of memory, which depending on memory fragmentation inside the process, may fail far earlier than exhausting available memory. This sort of problem gets relieved only by having a lower fragmenting heap or larger memory space of native 64bit. Memory-based recycling is not a good resolution to this issue. WOW64 is not a good resolution either.
To me, it seems that you got the 64bit machine to have access to larger memory address space to deal with your application's needs. Thus, you need to look into running pure 64bit instead of trying to make WOW64 work - because WOW64 is ultimately a compatibility layer, not an architectural layer to bet the future on.
Incidentally, this is the same sort of problem that drove Microsoft.com to quickly move its website to x64 machines and all web applications to be pure 64bit or .Net 2.0... to reap the full benefits of 64bit computing.