Some reasons not to do anything scary in your DllMain

re: Some reasons not to do anything scary in your DllMain

Tue, 27 Jan 2004 15:11:00 GMT

I notice the errors themselves weren't checked for expository purposes :)

// error checking deleted for expository purposes
hinst = LoadLibrary(you);
hicon = LoadIcon(you, MAKEINTRESOURCE(5));
FreeLibrary(hinst);

re: Some reasons not to do anything scary in your DllMain

Shane King — Wed, 28 Jan 2004 00:56:00 GMT

Sometimes doing these kind of "immoral" things is the only way though. Several times I've had to do ill advised things in DllMain because I absolutely need to ensure the dll doesn't load if some condition isn't met. It seems rather unfortunate that the only mechanism provided to do this shouldn't (in theory) be used to do it.

re: Some reasons not to do anything scary in your DllMain

Dan Maas — Wed, 28 Jan 2004 01:30:00 GMT

Is this "loader deadlock" responsible for the phenomenon where Windows gets into a state where you can't start any applications? (double-click and they just do nothing)

re: Some reasons not to do anything scary in your DllMain

Mike — Wed, 28 Jan 2004 02:29:00 GMT

Shane: Can you provide examples? I'd be interested in knowing any loader problems that can't be worked around.

Dan: On Windows NT, the loader deadlock issue is local to the process. Each process has its own loader lock.

re: Some reasons not to do anything scary in your DllMain

Peter Torr — Wed, 28 Jan 2004 04:38:00 GMT

Dan, you have probably run out of desktop heap. Happens with 30-ish IE windows open (or a similar number of other apps):

http://support.microsoft.com/default.aspx?scid=http://support.microsoft.com:80/support/kb/articles/Q126/9/62.ASP&NoWebContent=1

re: Some reasons not to do anything scary in your DllMain

Larry Osterman — Wed, 28 Jan 2004 05:14:00 GMT

Dan: Yes, the loader lock IS responsible for some of those situations. Especially on Win9x.

And even more to the point, it turns out that many of these situations are undebuggable - the problem is that the debuggers can't break into the app because to break into the app, they need to create a thread, which means that the loader lock needs to be taken (to call the DllMain in each DLL to tell them that the thread was created), but that hangs because someone has the loader lock held and hasn't released it.

The most heinous example of this is DllMain functions that do:

DllMain(<whatever)>)
{
if (ulReason == DLL_PROCESS_ATTACH)
{
CreateThread(ThreadToInitializeStuff);
WaitForSingleObject(EventThatsSetWhenThreadRuns);
}
:
:
}

This is a recipe for deadlocks, unfortunately there's way too much code that does this.

One nasty problem is that this SOMETIMES works - not always, and not on all versions of Windows.

re: Some reasons not to do anything scary in your DllMain

Shane King — Wed, 28 Jan 2004 05:59:00 GMT

One scenario is where I'm loading the dll as a CLR profiler, which allows me to modify .NET code on the fly for tracing and debugging purposes.

The profiler needs to have the chance to load on the startup of every .NET program, because you can't control the environment variables that cause the CLR to choose to laod it for a dllhost based process. Therefore, you must modify the global environment variables to cause it to always load.

The problem is, for most processes, I don't want it to be loaded. So I have to deliberately cause a failure result from DllMain to cause it to not be loaded and held in memory by the CLR. This is being determined via the registry, which is apparently a no-no in DllMain.

I'd love for there to be another way (particuarly since returning failure from DllMain causes nasty event log messages about not being able to load the profiler), but if you return failure from the initialization function of the profiler, it holds it in memory anyway (which is unnaceptable, as it makes replacing the dll with a new version needlessly difficult, as you'll need to shut down every process in which the CLR is active. This is particuarly a pain when developing the component, as Visual Studio .NET contains managed code).

Probably not the most common case in the world, but it's one where the DllMain approach seems to work well. Since it's basically used for debugging problems on production type machines where installing a real debugger is unacceptable, I don't particuarly care if a future version of windows breaks it. It's not like it's going to hurt anyone.

re: Some reasons not to do anything scary in your DllMain

Iain — Wed, 28 Jan 2004 06:45:00 GMT

http://blogs.msdn.com/cbrumme/archive/2003/08/20/51504.aspx

re: Some reasons not to do anything scary in your DllMain

Pavel Lebedinsky — Wed, 28 Jan 2004 07:35:00 GMT

>it turns out that many of these situations are undebuggable - the problem is that the debuggers can't break into the app because to break into the app, they need to create a thread...

This was definitely a problem with VC6 (I don't know if it was fixed in later versions). Fortunately, windbg/cdb can handle this just fine - they eventually time out and attach "non-invasively" (or you could use -pv option to force non-invasive attach to begin with).

re: Some reasons not to do anything scary in your DllMain

Doug — Wed, 28 Jan 2004 19:18:00 GMT

OCXs create this problem. VB is quite willing to unload an OCX while it is still running. So, the DLL's get unloaded, but the process is not in termination.
It is a great way for a thread to wake up and land in code space that is no longer loaded.

You can make a self reference to the dll so that it won't unload, but that can create other problems.

Personally, I think the bug is unloading of a DLL/OCX during runtime, but the VB development environment is quite willing to do it. If Microsoft does it, it must be correct.... grin.

re: Some reasons not to do anything scary in your DllMain

Norman Diamond — Thu, 29 Jan 2004 00:47:00 GMT

Mr. Chen, speaking of the MSDN page which Mr. Chen linked to, http://msdn.microsoft.com/library/en-us/dllproc/base/dllmain.asp

Here's another example where MSDN has recursion instead of useful information: "DllMain is a placeholder for the library-defined function name. You must specify the actual name you use when you build your DLL. For more information, see the documentation included with your development tools."

I don't understand "DllMain is a placeholder for the library-defined function name" because whenever I've seen DllMain the name has always been DllMain. Therefore I tried to obey "For more information, see the documentation included with your development tools."

Of course I first saw those instructions in documentation that came on CD instead of seeing them on Microsoft's web site. Guess what documentation is included with my development tools.

re: Some reasons not to do anything scary in your DllMain

Pavel Lebedinsky — Thu, 29 Jan 2004 06:59:00 GMT

>I don't understand "DllMain is a placeholder for the library-defined function name" because whenever I've seen DllMain the name has always been DllMain.

Typically, the real entry point is provided by CRT and is called something like DllMainCRTStartup. This function is the "real" "DllMain" as far as the loader is concerned. It performs CRT-specific initialization then calls your DllMain.

This is by the way the reason why global C++ constructors have the same limitations as DllMain.

re: Some reasons not to do anything scary in your DllMain

Norman Diamond — Fri, 30 Jan 2004 02:39:00 GMT

Mr. Lebedinsky, thank you for your explanation of DllMainCRTStartup but I'm still confused by MSDN. I think the MSDN page on DllMain is intended to instruct a DLL programmer on how to write a DllMain in VC++. I don't think that MSDN page is intended to instruct a CRT programmer on how to make a CRT support the writing of DLLs in VC++.

For comparison, if MSDN has a page on coding a main() (I'm neglecting to check if it does), I don't think it instructs CRT programmers on how to initialize the CRT and call the C program's main().

(Meanwhile the recursion in that MSDN page is still there and it's about to overflow my stack.)

re: Some reasons not to do anything scary in your DllMain

Mike Dimmick — Fri, 30 Jan 2004 15:09:00 GMT

When the DLL is initialised, the loader jumps to the address in the AddressOfEntryPoint member of the IMAGE_OPTIONAL_HEADER for the file. With MS tools, you can specify this with the linker's /ENTRY option.

If you don't specify /ENTRY, the MS linker defaults to _DllMainCRTStartup if you've also specified /DLL. This function (as Raymond alludes to on the other thread) has an explicit reference to the function name DllMain. It's always statically linked to your binary, even if you're using the DLL version of the CRT (it would be a chicken/egg situation otherwise). The CRT has a special weak-reference to an internal do-nothing DllMain so that your DLL still links even if you don't provide a DllMain of your own.

MSDN does have a page on WinMain, although it doesn't cover wWinMain. wWinMain is a fake provided by the CRT which uses GetCommandLine to retrieve the Unicode command line and pass it as the lpCmdLine parameter. main and wmain are also wrapped up by corresponding CRTStartup functions in the CRT, to present a WinMain-compatible entry point for the loader.

Indeed, all of the [w][Win]{M|m}ainCRTStartup functions share the same code, implemented in crt0.c (which you can read if you've installed the CRT source code).

If you have to specify your own entry point but still need the CRT (maybe you need to initialise something before the CRT starts up), call _cinit to initialise the CRT.

A peculiarity of Windows CE is that it has no wWinMain; instead, WinMain's lpCmdLine parameter is a LPWSTR. CE has no need for ANSI backward compatibility, of course. For CE, the CRT is part of the core system DLL, coredll.dll, which also includes all of the (supported) system entry points from the desktop's kernel32.dll, advapi32.dll, user32.dll, gdi32.dll and some others. The xxxCRTStartup functions are implemented in corelibc.lib.

re: Some reasons not to do anything scary in your DllMain

Norman Diamond — Mon, 02 Feb 2004 00:08:00 GMT

1/30/2004 7:09 AM Mike Dimmick:

Thank you for the detailed explanation of DllMain and [w]WinMain. When I have a free hour or two I might indeed look at the source of crt0.c. But actually the part of your posting I found most important was this:

> This function [...] has an explicit
> reference to the function name DllMain.

So my function still really has to be named DllMain(), and the MSDN page about DllMain might benefit by deleting threee sentences.

Now for one tangent.

> CE has no need for ANSI backward
> compatibility, of course.

For one application I accomplished that happy result by writing "\0xFE\0xFF" at the beginning of every text file. Then Pocket Word was able to display the Unicode contents of the text files. Otherwise I would have had to convert all strings from Unicode to Shift-JIS (ANSI code page 932) before writing them to text files.

re: Some reasons not to do anything scary in your DllMain

Raymond Chen — Mon, 02 Feb 2004 01:07:00 GMT

The point is that from the Platform SDK's point of view, the function name can be anything, and it's up to the language runtime provider to decide what to call it.

It so happens that the VC people decided that the name of the function is... DllMain! This is something that should be called out in the VC docs; the Platform SDK is compiler-agnostic. For example, if you write a DLL in assembly language (which I have done), then your DLL entry point can indeed be called anything you want, as long as you tell the linker.

Arguably the VC people should have called the entry point function something like CRTDllMain. But then again, most people don't understand the difference between the platform and the runtime. (That's part of the problem we're having here. The Platform SDK says the name can be anything, but the runtime says the name must be DllMain.)

re: Some reasons not to do anything scary in your DllMain

Norman Diamond — Mon, 02 Feb 2004 08:30:00 GMT

2/1/2004 5:07 PM Raymond Chen:

> Arguably the VC people should have called
> the entry point function something like
> CRTDllMain.

_DllMainCRTStartup is something like CRTDllMain, no problem for me on this point.

> most people don't understand the difference
> between the platform and the runtime.

Bingo. If Microsoft's development group in its internal operations finds it convenient to architect these separately, no problem. But why should customers have to distinguish them? From the point of view of ordinary programmers, a function call results in calling a function in some library which usually result in system calls somewhere down the line. There are reasons for ordinary programmers to discover that MFC is separate from the API, but why should we have to know or care which parts of the API are which?

If Visual Studio could still be used for programming for other than Microsoft OSes then I could see your point. (Yeah I see comments in some header files about targeting Macs, but the Visual Studio IDE doesn't provide that target.)

re: Some reasons not to do anything scary in your DllMain

Raymond Chen — Mon, 02 Feb 2004 15:23:00 GMT

The distinction between the Platform and the runtime is critical for people programming in languages other than C/C++ or for people using a compiler different from Visual Studio. If the Platform SDK blurred the distinction, then people using, say, the Borland compiler, would complain furiously that "Microsoft is playing unfair, writing Platform documentation that works only with its own compiler", and they would have a valid point.

re: Some reasons not to do anything scary in your DllMain

Norman Diamond — Tue, 03 Feb 2004 00:50:00 GMT

2/2/2004 7:23 AM Raymond Chen:

> The distinction between the Platform and the
> runtime is critical for people programming
> in languages other than C/C++

Since there's no Platform SDK for VB, I can see that VB programmers have to be aware that they only have the Runtime. But the VB Runtime is already different from the VC++ Runtime. So I still don't understand why the distinction is critical.

> or for people using a compiler different
> from Visual Studio.

Do you mean that Borland includes its own C++ Runtime? OK, if the difference between Platform and Runtime is critical for people who use a Runtime different from Visual Studio, then I understand. To achieve this understanding, it is critical to observe the distinction between the Compiler and the Runtime :-)

(From the point of view of the standard, it isn't critical to observe such distinctions. An implementation includes a preprocessor and compiler and library and everything. There is no guarantee that you can mix a compiler and half of a library from one implementation with the other half of a library from another implementation and come up with anything usable. But the standard isn't the most relevant thing here.)

re: Some reasons not to do anything scary in your DllMain

Raymond Chen — Tue, 03 Feb 2004 05:34:00 GMT

People programming in VB or C# often will use a p/invoke-style mechanism to call OS functions, so they need to know whether a particular function is OS (provided with Windows) or whether it's runtime (doesn't come with Windows; comes with the corresponding language).

For example, you can't p/invoke std::wstring.

And yes, different compiler vendors have different runtime libraries. From a Standards point of view, the world "beneath" the program is monolithic. This is a valid view for the Standard to take (since it's not in the business of dictating how OSs interface with languages), but it's not how the world actually works.

re: C scoped static initialization is not thread-safe, on purpose!

The Old New Thing — Tue, 09 Mar 2004 21:11:00 GMT

SimpleScript Part One: DllMain is Boring

Fabulous Adventures In Coding — Thu, 01 Apr 2004 09:02:00 GMT

Never, never ever call TerminateThread

Martin's WebLog — Sat, 06 Nov 2004 12:24:00 GMT

Sorting It All Out : Honey, you are the [_tWin]Main source of joy in my life!

Sorting It All Out : Honey, you are the [_tWin]Main source of joy in my life! — Thu, 24 May 2007 15:41:46 GMT

PingBack from http://blogs.msdn.com/michkap/archive/2007/05/24/2840970.aspx