-
Yup, as you all may know by now,
blogs.gotdotnet.com will go into the deep freeze very soon, so all new posts (there
will be new posts, I promise) will be available at http://weblogs.asp.net/oleglV
-
Yes, I admit it - I have been bad.
I haven't posted in quite a while - although I do have quite a few ideas for the new
posts. Not to get into the lame details, but lots of things have been going on in
just about every area of my life which kept me quite thoroughly occupied. Some of
those have been work-related and I actually have more material to post now...
Anyhow, things appear to be mostly improving and I expect to start posting in the
nearest future...
just as soon as I manage to defrost ( it has been pretty cold down here... )
-
So... if a tree falls in the forest, and no one is there to hear it, does it make a sound? Very new and original question, eh? Do I know the answer? Does anyone? Does this have anything to do with custom attributes? You'd be surprised to know that yes, in fact it does. But fortunately - as far as attributes are concerned - I do know the answer. Sort of.
This entry was supposed to be introductory. In fact, I wanted it to be very short and neat, just to start off with something simple. Well, as I started to review my notes, I realized that I will have no such luck.
As we know, attributes - more specifically custom attributes - are an extensibility mechanism which makes it possible to add bits of custom metadata in a non-intrusive way. This metadata is stored is a special metadata table which can then be looked at by various tools via Reflection or Metadata API. Attributes are extensively used by compilers, validation tools, execution environments, and parts of CLR execution engine - for instance Remoting and Interop subsystems. They are great, because they don't get in your way if you don't want them to and right there if you are interested.
So... let's have a closer look and see what exactly happens when write something like this:
[AttributeUsage( AttributeTargets.Class )]
public class ClassDescriptionAttribute : Attribute
{
public ClassDescriptionAttribute( string description, bool localizable ) {...}
public string Description {...}
public bool Localizable {...}
public string ResourceID {...}
}
[ClassDescription("Class Foo", true, ResourceID = "id1") ]
[Serializable]
public class Foo
{
}
As you can see, everything looks pretty straight-forward. There is a custom attribute called "ClassDescriptionAttribute" (I have omitted method bodies and private members because they are pretty standard - primitive getters and setters ); and a class, which I have inventively called "Foo" that has two attributes associated with it - the custom one we have declared, and the standard system attribute "Serializable", which is commonly used by Remoting subsystem. The semantics of what's going on are pretty clear as well - we want class Foo to have a localizable description with resourceID="id1". Something in our application will then extract this information and use it accordingly - the typical use would be to display it together with the other class information.
Let's compile this thing, ILDASM it and see what declaration of Foo looks like in IL.
.class public auto ansi serializable Foo extends [mscorlib]System.Object
{
.custom instance void Attribs.ClassDescriptionAttribute :: .ctor (string, bool) =
( 01 00 09 43 6C 61 73 73 20 46 6F 6F 01 01 00 54 0E 0A 52 65 73 6F 75 72 63 65 49 44 03 69 64 31 )
// ...Class Foo...T..ResourceID.id1
...
}
Huh, so this is interesting. There are several puzzling things there, which we will try to figure out one by one.
1. Where did "Serializable" go? On pseudo-attributes
Very good question. If you look closer, you will see some very explicit signs of presence of our freshly-defined custom attribute, but [Serializable] is not there. Well, turns out, that's because SerializableAttribute is what they called "pseudo-attribute", which means that it's not a "real" attribute, but something the C# compiler recognizes and translates into a class flag "serializable". Make no mistake, there is actually a class called "SerializableAttribute" that is defined in mscorlib.dll - it's just that it never gets emitted by the compiler as such.
The reason for this is pretty obvious -extraction of custom attributes is rather expensive, so CLR execution engine tries to avoid doing that as much as possible. Having "serializable" as a class flag as opposed to a "real" custom attribute, significantly improves performance of the Remoting subsystem.
There are other pseudo-attributes, full list of which you can find here. There are two main things to remember about pseudo-attributes
· They are very special and mostly intended for CLR use - as you see there're only a handful of them, and adding new ones would require CLR metadata specification change
· They can not be extracted using Type.GetCustomAttributes(), instead they need to be accessed via appropriate metadata item properties or Metadata API
2. What's that ".ctor" business? What custom attributes really are
As you can see, .custom statement - you can probably guess that it used to declare custom attributes in IL - mentions ClassDescriptionAttribute in rather peculiar way - it seems that it is trying to "call" something called ".ctor". Why is that? As you may guess, ".ctor" means "constructor"(incidentally, ".cctor" means "static constructor"). Intuitively, this makes sense, as an attribute class can have more than one constructor, so we have to be specific as to which one we want "called".
Things get a little clearer if you consider how constructors are represented in CLR Metadata tables. You see, all constructors are stored together with other methods in MethodDef/MethodRef tables, and while certain flags do indicate quite clearly that this particular method is in fact a constructor, these are the only two tables that contain information about them.
Another Metadata table that is of interest to us is called CustomAttribute. This one - as you may have guessed - describes custom attributes. This particular table has 3 fields - Parent, Type and Value. Parent is something that the attribute is defined on (assembly, class etc); Value is that ugly blob we'll talk about later, and Type points to another metadata item. CLI demands that that metadata item should be stored in MethodDef/MethodRef tables and be an instance constructor. (Incidentally there are some indications that Value could, in fact, point to something else - current metadata token encoding allows for that. Whether that will ever be exploited remains to be seen...)
So, based on all of the above, a custom attribute is an association between any metadata item (except another custom attribute) and a instance constructor. At least, this is what it is based on the CLI as per ECMA-335. That's why we see ".ctor" explicitly mentioned in IL declaration - it is, in fact, just a pointer to an entry in MethodDef table.
3. What's that binary goop there? On custom attribute value encoding.
So what in the world is that scary blob that is being "assigned" to the constructor? It's clear it has something to do with attribute initialization - you can kind of see some hints of that in the character dump of the blob - but what exactly?
Well, as you would expect, that is information necessary to initialize the attribute class properly. This is not - as one may assume - a binary in-memory representation of the attribute class instance itself, but instead encoded constructor arguments and property name/value pairs, allowing for subsequent attribute initialization.
You can get a very detailed information on how to read this blob in Serge Lidin's Inside IL Assembler, but in a nutshell, here's what this it says
· 01 00 - prolog
· 09 43 6C 61 73 73 20 46 6F 6F - first ctor argument; string "class Foo" prefixed by its length (0x09).
· 00 - second ctor argument; bool "false"
· 54 0E 0A 52 65 73 6F 75 72 63 65 49 44 03 69 64 31 - name/value pair representing "ResourceID"="ID1".
0x54(SERIALIZATION_TYPE_PROPERTY) tells us this is a property name/value pair
0x0E(ELEMENT_TYPE_STRING) specifies property type
0x0A is the length of "ResourceID";
0x03 is the length of "id1"
As you can see, while the constructor argument types are extracted from its signature and not explicitly encoded in the blob, property name/value pairs explicitly specify property/field type.
So that blob represents a sequence of actions that need to be carried out to create an attribute instance. Who gets to create an attribute class? Read on...
4. What was that bit on proverbial trees about? Who really creates a custom attribute.
So as we can see, there is more than enough information to create an instance of ClassDescriptionAttribute - we know the class, necessary constructor signature, its arguments and additional properties to assign.
So what does it get created by? The compiler? It actually emits all the information necessary to create tan instance of the attribute class, but no, it doesn't create it in a conventional sense.
CLR execution engine? That's a possibility, but how would that work - would it just go ahead and create all custom attributes on startup? Keep in mind, we actually need execute some IL code to create an instance of an attribute class - namely the constructor - surely doing all that would be an awful waste of time, especially if no-one bothers to actually access the attributes?
You have probably guessed it - instances of attribute class are actually created by Reflection when someone asks - typically via Type.GetCustomAttributes(). I suspect that this is considered to be an implementation detail of the Reflection API though - all you really need to know is that your attribute class gets created, and you can't really control when and how.
While we are on the subject of implementation details, you will find that - at least in .Net v1.1 - attribute classes are in fact created every time you ask for them.
Let's update the constructor of ClassDescriptionAttribute as follows:
public ClassDescriptionAttribute( ... )
{
Console.WriteLine( "ClassDescriptionAttribute created");
...
}
And then run the following code snippet:
Console.WriteLine("Extracting custom attribute...");
ClassDescriptionAttribute descriptionAtribute = typeof(Foo).GetCustomAttributes( typeof( ClassDescriptionAttribute ), false )[0] as ClassDescriptionAttribute;
Console.WriteLine("Extracting custom attribute yet again...");
ClassDescriptionAttribute descriptionAtribute2 = typeof(Foo).GetCustomAttributes( typeof( ClassDescriptionAttribute ), false )[0] as ClassDescriptionAttribute;
Console.WriteLine( descriptionAtribute == descriptionAtribute2 );
Basically we are extracting the same attribute twice and then compare that the objects that get returned are in fact the same. If you run this code, you may be surprised to find out that it prints:
Extracting custom attribute...
ClassDescriptionAttribute created
Extracting custom attribute yet again...
ClassDescriptionAttribute created
False
Interesting, huh? Every time you call to ask for an attribute, Reflection creates another instance of it, which of course means that returned values will have different identity (by the way, you can make them "look" the same by redefining operator "=" and methods "Equal" and "GetHashCode" based on attribute fields rather then binary identity).
So let's get back to the original question - "does a custom attribute get created if no-one asks for it?". The answer is "no". There is a bit of a twist as far as pseudo-attributes are concerned, but then again those things never get created, so the answer still remains.
-
I think this is
pretty fantastic. And scary in a way. In fact I have been thinking something along
those lines for a while, just was never able to clearly articulate it.
I guess it just confirms the fact
that nothing happens by itself and that sometimes we are "so busy doing our job, we
forget to do our job" - in a larger sense, anyway...
I guess it all comes down to this
- why have you chosen to be a developer? What does it mean for you to be one? Sure,
our direct responsibility is to write code, get it to a good quality and ship it.
The very feeling that you have helped someone is pretty powerful as of itself, but
the reality is, there needs to be more to this. Eventually every person needs to ask
themselves this very selfish question "what's
in it for me?".
And that is a totally valid question
to ask - sometimes work can be tedious, exhausting and demanding, so there must be
a rewarding part somewhere. And I'm not talking about a delayed gratification of shipping
- no-one ships once a month, so there should be something that keeps you going. The
mere "I just like writing code" almost never covers all of it. There's always more
to it. You may like to learn new things, to work in a certain environment, to earn
certain income, to gain recognition ... or all of the above. But it is
important that everyone understands what it is that is really driving them and if
there is anything that is driving them - otherwise they may find it very hard to go
that extra mile we all have to go every now and then.
And it's trivial, but sometimes
we loose it somehow - a career is not something that just happens to you all by itself,
even if you work for a major company and steadily climb the ladder. More importantly,
career is not something objective - of two people doing the same job, one can be deliriously
happy and the other quite miserable.
I suppose, while I'm on the topic,
I should probably say what it means to me. I completely agree with Eric there
- it's all about "cluefullness" for me. I need to know that I have learned more, that
I have a broader and deeper view of the world. Why does it matter to me that the CLR
custom attributes are a little funny? That you
shouldn't be using DllMain in MC++ DLLs if the current release of CLR and why?
I have no idea. It just does. It makes me high. It makes me feel empowered... and
of course, it also helps me with my everyday work.
PS It goes without saying... but
let me say this anyway:
The content of this site are my
own personal opinions and do not represent my employer's view in anyway. In addition,
my thoughts and opinions often change, and as a weblog is intended to provide a semi-permanent
point in time snapshot you should not consider out of date posts to reflect my current
thoughts and opinions.
-
Last
time I was talking about DllMain and what nasty things can occur if you misuse
it. I have also mentioned that it may not be one of those "I'm always careful it can
never happen to me" situations - things can get out of hand very quickly.
Keep in mind that OS loader has
been evolving over time. OS creators know that not all DLLs are well-behaved and they
have been trying to do their best to minimize the impact of poorly-written DllMains,
however it is still more than possible to shot oneself in the foot.
Let me give you a very simple example
as to how easy this can be (behavior may vary on different OS's, I'm running this
on Windows XP SP1).
Consider the following:
/////////////////////////////////////////////////////////////////////
Dll2.cpp
/////////////////////////////////////////////////////////////////////
HMODULE g_Module;
TCHAR g_tclpszFileName[256];
BOOL APIENTRY DllMain( HINSTANCE
hModule,
DWORD ul_reason_for_call,
LPVOID
lpReserved
)
{
if (
DLL_PROCESS_ATTACH == ul_reason_for_call )
{
printf("Dll2:DllMain\r\n");
g_Module
= hModule;
::GetModuleFileName(
g_Module, g_tclpszFileName, 255 );
}
return TRUE;
}
extern "C" __declspec(dllexport) void WINAPIV
OutputModuleInfo2(void)
{
printf("Enetering
Dll2::OutputModuleInfo2\r\n");
printf("Name:
%s\r\nHandle 0x%x\r\n", g_tclpszFileName, g_Module );
}
/////////////////////////////////////////////////////////////////////
// Dll1.cpp -
NEVER do this
/////////////////////////////////////////////////////////////////////
typedef void (WINAPIV
*LPFOUTPUTMODULEINFOFUNC) (void);
HMODULE g_Module;
TCHAR g_tclpszFileName[256];
BOOL APIENTRY DllMain( HINSTANCE
hModule,
DWORD ul_reason_for_call,
LPVOID
lpReserved
)
{
if (
DLL_PROCESS_ATTACH == ul_reason_for_call )
{
printf("Dll1:DllMain\r\n");
g_Module = hModule;
::GetModuleFileName(
g_Module, g_tclpszFileName, 255 );
//
Load Dll2 - never EVER do this
HMODULE
hModule1 = ::LoadLibrary("Dll2.dll");
LPFOUTPUTMODULEINFOFUNC
lpOutputModuleInfo1Func = (LPFOUTPUTMODULEINFOFUNC)::GetProcAddress( hModule1,"OutputModuleInfo2");
lpOutputModuleInfo1Func();
}
return TRUE;
}
extern "C" __declspec(dllexport) void WINAPIV
OutputModuleInfo1(void)
{
printf("Enetering
Dll1::OutputModuleInfo1\r\n");
printf("Name:
%s\r\nHandle 0x%x\r\n", g_tclpszFileName, g_Module );
}
/////////////////////////////////////////////////////////////////////
// Main.cpp
/////////////////////////////////////////////////////////////////////
extern "C" __declspec(dllimport) void WINAPIV
OutputModuleInfo1(void);
extern "C" __declspec(dllimport) void WINAPIV
OutputModuleInfo2(void);
int _tmain(int argc,
_TCHAR* argv[])
{
OutputModuleInfo1();
OutputModuleInfo2();
return 0;
}
What's wrong with this? Let me
count the ways. On top of non-existent error-handling and the fact that we have an
un-paired LoadLibrary() call, this code has a very fundamental problem. Let's
just say that depending on how this code is compiled, it may
-
Run and produce results you expect
-
Run and produce results you don't
expect
-
Blow up with AV
That's right.
Let's dig into it.
First let's see what this code
is supposed to do in the first place. You'll have to bear with me here - it's after
midnight and I haven't been able to come up with something brilliantly meaningful,
but this will just have to do for now.
As you can see, we are dealing
with two DLLs and one EXE that uses those DLLs. First DLL (inventively called Dll2)
- gets its own HMODULE in DLLMain(DLL_PROCESS_ATTACH), gets its name based on that
and stores them away in global variables. Exported function OutputModuleInfo2 simply
prints that out using printf.
Dll1 is almost identical, except
it dynamically calls into Dll2 right after collecting its own information. It's a
little weird, but this is just a primitive example after all.
Main
is a console appliccation that
is statically bruit against export libraries produced by the build of the first two
DLLs and calls both OutputModuleInfo1 and OutputModuleInfo2.
Simple enough? Let's roll.
-
It
works! It works! Let's
compile everything, but make sure that all three binaries use CRT(C/C++ runtime) dynamically
(/MD compiler option) and that Dll2.lib appears before Dll1.lib in linker options
pertaining to additional input libraries for our console app (something like link.exe
main.obj /out:main.exe dll2.lib dll1.lib). Turns out that is important - we'll
see why in a little bit. When you run the application, it outputs:
Dll2:DllMain
Dll1:DllMain
Enetering
Dll2::OutputModuleInfo2
Name:
c:\Temp\KillDllMain\Dll2.dll
Handle
0x10000000
Enetering
Dll1::OutputModuleInfo1
Name:
c:\Temp\KillDllMain\Dll1.dll
Handle
0x320000
Enetering
Dll2::OutputModuleInfo2
Name:
c:\Temp\KillDllMain\Dll2.dll
Handle
0x10000000
As you see, things seem to be working
fine. One thing that is worth pointing out is that - as you can see from the output
- DllMain for Dll2.dll was called before DllMain of Dll1.dll. Why? Well, technically,
there's no explicit guarantee as to the order of these things - it is all in the loader's
hands. As
I mentioned before, the loader looks at static dependencies and builds a list
of DllMains to be called based on that. But what happens if the order really doesn't
matter? From loader's perspective, Main.exe depends on Dll1 and Dll2 and there's no
reason to choose one over the other (remember, the fact that Dll1 does in fact load
Dll2 is our little dirty secret).
Well,
turns out that the loader seems to be preserving the order in which the imported DLLs
are listed in the Imports Section of the loading executable. You can read all about
the low-level details in Matt
Pietrek's article, but for the purpose of this discussion let's just say that
each PE file (EXE or DLL) knows what binaries it "references" - that is what external
functions it imports - and that a list of those binaries, together with referenced
functions is linked into its PE header. Microsoft Linker seems to build that header
based on the order in which export libraries are supplied, which is why we built our
app the way we did.
What
happens if change that? Let's see.
-
Huh?
But... So now let's
build the same code, only this time supply export libraries in the opposite order
(something like link.exe main.obj /out:main.exe
dll1.lib dll2.lib). Let's run it:
Dll1:DllMain
Enetering
Dll2::OutputModuleInfo2
Name:
Handle
0x0
Dll2:DllMain
Enetering
Dll1::OutputModuleInfo1
Name:
c:\Temp\KillDllMain\Dll1.dll
Handle
0x10000000
Enetering
Dll2::OutputModuleInfo2
Name:
c:\Temp\KillDllMain\Dll2.dll
Handle
0x320000
Interesting... As you see, this
time DllMain from Dll1 got called first. That loaded Dll2 and its OutputModuleInfo2
got called ... before its DllMain! No wonder it printed what it did. Note that the
second call into OutputModuleInfo2 went through just fine because Dll2's DllMain was
called already.
So why in the world is OS loader
acting so dumb? Doesn't it know we are loading Dll2? We have explicitly called LoadLibrary
after all, which loaded it from disk, laid it out in memory, resolved its exports
etc. Why wasn't DllMain called? If you experiment a little, you will find out that
in most cases DllMain of dynamically loaded libraries will be
called, even if the "illegal" LoadLibrary is used to load it. The only case that will
not take place is when OS loader already "knows" about that DLL but hasn't yet called
DllMain on it, which is exactly what happened here.
Main.exe statically depends
on Dll2.dll, so it's already in the loader's plan. It turns out, the loader is not
so willing to change its original plan created based on static dependencies. If new
binaries get thrown in, the loader will stop and dutifully load them; but if the binary
is in fact the "old" one - that is already in the plan - the loader will just skip
it.
Why? My guess is that this works
pretty well for most scenarios. The loader is still trying to be nice and compensate
for our bad behavior. Once we attempt to load something it already knows about, it
simply preserves its current plan - I suspect doing otherwise would cause all kinds
of nasty consequences. Mind you, we are
on no position to complain - we are not supposed to call LoadLibrary from DllMain
in the first place. Keep in mind, these are my speculations - I'm not trying to give
a precise recipe as to how the OS loader can be mistreated, I'm just saying that it
can be done.
So...there you go. In this particular
situation you "just" got the wrong value printed out, but you can imagine that this
can easily cause a wide range of nastiness - AVs for instance. Speaking of which...
-
What???
How did that happen?... Let's
build the whole thing again, only this time let's use static CRT (/MT or /ML compiler
options). Why should it matter, right?
Now let's run it:
Dll1:DllMain
...
and then... whoa...
First-chance exception at 0x77f57bd2
(ntdll.dll) in MainApp.exe: 0xC0000005: Access violation reading location 0x00000010.
But
why? If you look at the stack, you will see the following:
ntdll.dll!_RtlAllocateHeap@12() +
0x24
Dll2.dll!_heap_alloc(unsigned
int size=0x00000018) Line 212 C
Dll2.dll!_nh_malloc(unsigned
int size=0x00000018, int nhFlag=0x00000000) Line
113 C
Dll2.dll!malloc(unsigned
int size=0x00000018) Line 54 + 0xf C
Dll2.dll!_mtinitlocknum(int
locknum=0x00000011) Line 251 + 0x7 C
Dll2.dll!_lock(int
locknum=0x00000011) Line 311 + 0x6 C
Dll2.dll!_lock_file2(int
i=0x00000001, void * s=0x00346b68) Line
267 + 0x9 C
Dll2.dll!printf(const
char * format=0x0034204c, ...) Line 57
+ 0xd C
Dll2.dll!OutputModuleInfo2() Line
30 + 0xa C++
> Dll1.dll!DllMain(HINSTANCE__
* hModule=0x10000000, unsigned long ul_reason_for_call=0x00000001, void * lpReserved=0x0012fd30) Line
30 + 0x5 C++
Dll1.dll!_DllMainCRTStartup(void
* hDllHandle=0x10000000, unsigned long dwReason=0x00000001, void * lpreserved=0x0012fd30) Line
297 + 0xd C
So this is caused by calling "printf"
from Dll2's OuputModuleInfo2,
which is sort of strange. If you look some more, you will find that the CRT internal
global _crtheap is NULL, which means that CRT has no heap. Why? You guessed it
- static CRT allocates its heap in DllMain of the owning DLL! If our case DllMain
wasn't called yet, so naturally - no heap.
Ouch. (Incidentally, this means
that just about any CRT call will AV - it's awfully difficult to do anything without
allocating any memory...)
Moral
OK, this is much longer than I
intended... but here's the moral: be
careful. OS loader is
not dumb, and it is as forgiving as it gets, but sometimes it won't be there to help
- simply because it has no idea what your intentions are.
OK, I think I'm officially done
with the topic - I'm feeling much better now :)
-
Preface
OS loader has always intrigued
me - probably because it works behind the scenes and no-one normally bothers to understand
what is that is does exactly, until strange or funny things start happening. And they
do. And then we read through the documentation and we are forced to remember that
there's more to loading a binary than just slapping it into process address space.
In fact there's
a wonderful article by Matt Pietrek that discusses those matters. I strongly encourage
every person who deals with native code to go and read it - it may be quite enlightening
for you - I know it was for me. When you know how things get loaded, you are less
likely to forget to re-base your binary, consider early binary binding etc.
Every now and then another piece
of information or a great summary on the subject comes up and I find myself mystified
with the whole loader topic all over again. This time it was a
very lengthy post in Chris Brumme's
blog. As many people have mentioned, the post in question is very long and very
dense with technical information well, what else did you expect from Chris's blog?
:) Anyway, in order to absorb the topic
better and in hopes of getting the whole thing out of my system I decided to write
things down.
DllMain
and OS loader
As we are all well aware now, things
are not as easy as they seem. In fact are they ever? DllMain which used to be briefly
discussed in most books on Win32 as a reasonably innocent initialization routine may
now look like a vicious monster which obeys no rules and causes nasty side-effects.
But let's get to the source - MSDN
reference
It all starts innocently enough.
The article defines DllMain as an optional entry point into a DLL, called by the system
when the DLL gets attached to a process or a thread; outlines the somewhat tricky
but reasonable rules that govern the calls (for instance, calls may be unmatched for
a thread if it's a main thread of the process or if it was already running when LoadLibrary
was called), discusses abnormal termination and then
...whoa...
Without missing a heart-beat, it
carries on describing what you can do there. That is pretty startling as of itself
since when should you be limited in that regard? - but as you keep reading, things
just get worse. It turns out, you can do pretty much nothing at all. Calls to LoadLibrary/LoadLibraryEx
are explicitly prohibited. Other calls into kernel32 are OK. But you can't call into
User32. And don't use CRT memory management (unless you are linked statically)
- use HeapAlloc instead. Oh, and of course don't call anything that would do any such
nasty things: that would be bad. One last thing - don't read the registry either.
Have a nice day.
The fact that none of this is written
is big, bold, maybe
even red print is truly
unfortunate - it really ought to be, because most people simply miss that part.
So let's say, you have read it all now the question is: why?
The thing is, as far as your binary
is concerned, DllMain gets called at a truly unique moment. By that time OS loader
has found, mapped and bound the file from disk, but - depending on the circumstances
- in some sense your binary may not have been "fully born". Things can be tricky.
In a nutshell, when DllMain is
called, OS loader is in a rather fragile state. First off, it has applied a lock on
its structures to prevent internal corruption while inside that call, and secondly,
some of your dependencies may not be in a fully loaded state. Before a binary gets
loaded, OS Loader looks at its static dependencies. If those require additional dependencies,
it looks at them as well. As a result of this analysis, it comes up with a sequence
in which DllMains of those binaries need to be called. It's pretty smart about things
and in most cases you can even get away with not following most of the rules described
in MSDN - but not always.
The thing is, the loading order
is unknown to you, but more importantly, it's built based on the static import
information. If some dynamic loading occurs in your DllMain during DLL_PROCESS_ATTACH
and you're making an outbound call, all bets are off. There is no guarantee that DllMain
of that binary will be called and therefore if you then attempt to GetProcAddress into
a function inside that binary, results are completely unpredictable as global variables
may not have been initialized. Most likely you will get an AV.
Another scenario is when you start
spinning a new thread on DLL_THREAD_ATTACH and wait for it to finish initialization
via some syncronization technique. This blocks your thread in DllMain, while still
keeping OS lock. This can lead to deadlocks.
Overall, if anything - anything -
goes wrong in DllMain of one of the binaries, the whole process may be doomed.
The trouble is, definition of "wrong"
is very, very vague in this case. For instance, developers using MC++ know that you
shouldn't even dream of having DllMain in your library. And if you do you do, you
may be very, very sorry. I think CLR folks want to fix this for the "Whidbey"
release.
Chris Brumme lists the following
things that should never, ever be done in
DllMain.
· Dynamic
binds. That includes
LoadLibrary/UnloadLibrary calls or anything that may call implicitly call them
· Locking of
any kind. If you are trying to acquire a lock that is currently help by a thread that
needs OS loader lock (which you may be holding), you'll deadlock.
· Cross-binary
calls. As been discussed
the binary youre calling into may not have been initialized or have already been unutilized.
· Starting
new threads and then wait for completion.
As discussed, thread in question may need to acquire OS lock that you are holding.
So, what does this tell us?
DllMain
is that gun you can easily shoot yourself with
How many people do you know that
did stupid things like calling CoInitialize() in DllMain? I know of cases when that
was done on DLL_THREAD_ATTACH, which not only means that we were risking to hit a
deadlock, but also that any thread in that process will have COM initialized. What's
worse, it may be initialized with the wrong threading model. And then people will
be wondering how the heck they ended up with STA threads in thread pools. Or something
much more subtle like calling a system function that starts a worker thread as part
of its execution? How many times did you do
all those things?
Another problem with this is that
all these horrors can present themselves under very limited circumstances. In most
cases things do work fine, but a race condition, a slightly modified DLL load order
or other factors may change everything. Which means you may not even know it until
your ship. This may be fine for a user application (well, things like that are never fine,
it's just that the damage may not be substabtial), but this is always bad for servers
- especially if you are talking enterprise availability. I don't think this can
ever become a security threat - one you can fight anyway - but random crashes
are just not nice.
So let's get back to what we can do
in DllMain. According to MSDN, "The entry-point function should perform only simple
initialization or termination tasks."
These tasks can only include calls
to Kernel32 (excluding LoadLibrary/LoadLibraryEx). If you look at what this means
for you, you will find that this is extremely liming. Further,
CRT functions, including memory allocations are not safe unless you are statically
linked. This means that seemingly innocent things something like g_pMyGlobalObject
= new CMyGlobalObject() can theoretically cause all kinds of nasty stuff because
they will use malloc that is dynamically
linked from msvcr*.dll.
This leaves us with primitive types,
synchronization objects initialization ... that's about it. And definitely - definitely -
no managed code.
So what am I saying? There aren't
too many things that are legal there; it's extremely easy to do illegal stuff - you
have to always know if what you're calling really does, which is extremely difficult
if you use something defined elsewhere - C/C++ LIB for instance; the compiler won't
tell you that you are doing the wrong thing; and the code is likely to run fine in
most cases... but not all of them.
Where options does this leave us
with?
-
Just
say no. Avoid the darn
thing altogether and link with /noentry. Reconsider the way you deal with globals.
Do lazy TLS initialization.
-
Be
very careful. Sometimes
you simply have to use it. It's just too ugly not to. Have a full code review. See
what's being done and what OS does. Make sure that everyone understands that DllMain
is just different. Read and memorize horror stories about people who didn't know better.
One thing you can do here to minimize the damage is disabling calls to your DllMain
when new threads join/leave the process - this can be done with DisableThreadLibraryCalls.
This is generally a good idea in all cases where you don't need thread-level initialization
because OS loader doesn't need to call into your binary every time a new thread is
born
-
Be
afraid. Be very afraid. Well,
just leave things where they are. Things don't crash right now and you have other
things to do. Good plan.
Silver lining
: DllMain and resource leaks diagnostics
There's one piece of information
that gets provided through DllMain which you can't possibly get any other way. If
you review the signature of DllMain, youll notice that the last argument passed in
despite being called lpReserved actually has some meaning:
If fdwReason is
DLL_PROCESS_ATTACH, lpvReserved is NULL for dynamic loads and non-NULL for
static loads.
If fdwReason is
DLL_PROCESS_DETACH, lpvReserved is NULL if DllMain has been called by
using FreeLibrary and non-NULL if DllMain has been called during process
termination.
As you see, lpvReserved does tell
you something. Although I can't see why you would be interested in knowing whether
your DLL has been statically or dynamically loaded - there may be uses there,
I just don't see them - but knowing how you are being unloaded could be interesting.
For one, if you're managing some
kind of resource in DllMain, which only lives within process context, you can possibly
skip some clean-up if you knew that the process is dying as it is. This is not too
valuable because the very nature of DllMain does not make it a very good entry point
for resource management.
There are cases, however, when
you expect your DLL to be unloaded in a specific way and you can use DllMain to verify
that it is indeed being unloaded as you expect. For instance, if:
· your
DLL is in fact a COM server (and has no other uses), and
· the
COM host is well-behaved and
· all
of your COM objects have been properly released,
then you should expect that you
will get lpvReserved=NULL - that is unloaded via FreeLibrary.
Heres what seems to be happening.
Every well-behaved COM process should call CoUnintialize() on each thread when it
gets shut down. Internally that calls DllCanUnloadNow on your binary which returns
TRUE if all outstanding references are closed. If that's the case, COM will call FreeLibrary,
which - unless there are other LoadLibrary references outstanding - will unload your
DLL. That will pass lpvReserved=NULL. If any of
these conditions is not satisfied, your DLL will reside in the process until it terminates
and you'll get lpvReserved!=NULL( I'd like to thank
Michael
Entin
- who really ought to start blogging - for helping me to get all the pieces together).
So if - and that's a big
if - your application
is well-behaved, and no-one ever messed up loading your DLL with LoadLibrary and forgetting
to unload it, then lpvReserved!=NULL means that some of your COM objects have not
been released. There's nothing your code can do about that - except maybe asserting
- and you will then have to look into that further.
This approach is not limited to
only COM leaks - theoretically you should expect that when your binary is leaving
this world, it's not taking anything with it. You can look through the list of globally-managed
resources and see of they have been disposed if. Be very, very careful there - you
shouldn't be doing any stuff that may compromise OS loader: see the four bullets
above.
-
It's late. I'm very hungry. I really
want to go home. In fact, I almost left 3 hours ago, except something very strange
started happening with our UI bits and I didn't know why. Turns out we had a plumbing
problem. And now I need to vent.
Plumbing is great. In fact it's
so great that you don't even know it exists until your lawn starts smelling funny
or your toilets stop flushing. And then you're forced to remember.
Windows Forms is a fantastic library
is a sense that it tries to isolate you from a lot of yucky stuff that you don't want
to deal with - handles, message pumping, lovely functions such as CreateWindowEx that
take a boatload of arguments etc.(that is on current version Windows of course. On
other platforms - or maybe future Windows platforms - the "yucky stuff" may be entirely
different. But there will be some).
Basically WinForms hide the plumbing, and it feels great... and then you pipe bursts.
Bizarrely, sometimes you didn't even know there was a
pipe. Worse yet, something you don't even know what the heck is happening, except
there's a whole lot of water on your carpet.
Enough of this - here's what happened.
Pop-quiz: what's wrong with this code?
public class Form1
: System.Windows.Forms.Form
{
private ToolBar
toolBar1;
private ToolBarButton
toolBarButton1;
public Form1()
{
InitializeComponent();
}
private void InitializeComponent()
{
this.toolBar1
= new System.Windows.Forms.ToolBar();
this.toolBarButton1
= new System.Windows.Forms.ToolBarButton();
this.SuspendLayout();
this.toolBar1.Buttons.AddRange(new System.Windows.Forms.ToolBarButton[]
{ this.toolBarButton1 });
this.toolBar1.ButtonSize
= new System.Drawing.Size(36, 36);
this.toolBar1.Name
= "toolBar1";
this.toolBar1.TabIndex
= 0;
this.toolBarButton1.Text
= "A";
this.Controls.Add(this.toolBar1);
this.Text
= "Form1";
this.Layout
+= new System.Windows.Forms.LayoutEventHandler(this.Form1_Layout);
this.ResumeLayout(false);
}
[STAThread]
static void
Main
()
{
Application.Run(new Form1());
}
private void Form1_Layout(object sender,
System.Windows.Forms.LayoutEventArgs e)
{
this.toolBar1.ButtonSize
= new Size( this.Width/10, this.Height/10
);
}
}
This is all pretty straight-forward,
huh? You have your form, which has a toolbox with one button. You then want your toolbar
buttons to resize appropriately, based on the size of the form, so you start handling
Layout event which, according to MSDN, is called "when a form or a control should
reposition its child controls" (yes, I realize that the handler sucks - this.Width/10
may turn in 0 and then things will go bad... just bear with me people).
Looks good? Good. Now run it. Seriously,
copy/paste this thing into a small C# project and run it. I'll wait.
I know. Such innocuous code and
such nasty error. If case you haven't gone through this little exercise, Application.Run()
throws Win32Exception saying "Error creating window handle". So now you have water
on your carpet and you don't know why.
There are three factors here
· Form.Layout
gets fired from ToolBar.OnHandleCreated().
Yup. It does. If you don't believe me, just set a breakpoint and look at the stack.
Why is that? I'm not certain. This is definitely a little excessive, but hardly is
a crime. All handles should be created by then, no reason to worry... or is there?
· In
Windows, toolbar button size can be set through TB_SETBUTTONSIZE message.
That's all great, but if you read MSDN carefully, you'll see that "The size can be
set only before adding any buttons to the toolbar".
Why is that? I really have no idea. Maybe it's just hard to resize things dynamically.
Again, a little too strict, but hardly a crime.
· Windows
Forms try hard to hide
all yucky stuff from you. So you add some buttons, call ToolBar.ButtonSize...
what a poor ToolBar to do? Easy - simply recreate the handle, and set the new size.
Again, a little excessive, but there isn't much that can be done here. OK, so it could
have thrown an obscure exception saying that you can't change button size, but that
would be too intimidating and confusing.
So far, so good. Let's put this
together(I haven't really seen source code for this, so this is all based on common
sense and debugging)
1. Your
run your app and the loop starts pumpin'
2. Your
form gets WM_SHOWWINDOW message and realizes
and needs to show its children, so it starts creating "real" Win32 windows and calls CreateWindowEx() for
everything, including the toolbar.
3. "Real"
windows toolbat gets created, WM_CREATE is
sent to it and OnHandleCreated() gets
called. Toolbar then fires up Parent.Layout.
Keep in mind we are still inside CreateWindowEx call
- it doesn't return until WM_CREATE is fully handled.
4. In Form1_Layout handler,
we attempt to change toolbar button size, which - as we know - can only be done via
handle recreation - so the toolbar calls DestroyWindow.
What does this mean? We
destroy the handle that was created while still in CreateWindowEx that creates the
very handle! Needless to say, Win32 gets upset, and CreateWindowEx fails.
Caboom.
So, is this a bug? I'd say Toolbox
should be much more careful with what it does - especially when that comes from OnHandleCreated.
Or make handle re-creation conditional based on whether buttons have been created
or not. Or both - that way Parent.Layout gets
fired first, then ButtonSize sees that there are no buttons yet, so no need re-create
the handle. Yup, that'll do it.
If only it was so easy. The truth
is, as much as you can try, when you attempt to create a simple and consistent model
on top of something that is not quite to straight-forward(Win32 is pretty old and
inconsistent in places, which is the very reason Longhorn API gets introduced), you
will always run into something like this.
[Sigh]
Anyway, all of this is fixable, but in a rather ugly way. This is what I came up with:
public class SafeToolBar
: ToolBar
{
private bool fullyCreated
= false;
internal bool FullyCreated
{
get { return this.fullyCreated;
}
}
protected override void OnHandleCreated(EventArgs
e)
{
base.OnHandleCreated
(e);
fullyCreated
= true;
}
}
I then use this class instead of
ToolBar, and modify my OnLayout as follows :
private void Form1_Layout(object sender,
System.Windows.Forms.LayoutEventArgs e)
{
if (
!toolBar1.FullyCreated )
return;
this.toolBar1.ButtonSize
= new Size( this.Width/10, this.Height/10
);
}
This way the "dangerous" part of
Form1_Layout will only be called after the toolbar is completely done creating.
I'll talk to WinForm folks and
see if we can permanently fix this...
[Sigh...again]Moral? The
plumbing is there. Even if your pipes are all neatly tucked away into white closets,
there's still water in them. It helps
to understand how things really work...
although in most cases it is convenient to forget the plumbing exists - if it works
well. And I definitely think that Windows Forms team has done a pretty great job hiding
it :)
P.S. I have discussed this with
Michael Entin - a colleague of mine (who really ought to start blogging... but I digress)
- and he went "Yeah, leaky
abstractions everywhere...". Read the link, it's nothing revolutionary, but very
insightful...
-
I keep changing my mind whether
C/C++ is a blessing or a cruel joke on us all. Sometimes I think it's great (when
it lets me cast a block of memory to just about anything I want), and sometimes I
really hate it(when it lets me cast a block of memory to just about anything I want). And
sometimes, I come across something that makes my jaw drop again.
Nothing new - just another
confirmation that C is just full of warts of many kinds.
This time I was reading Eric
Lippert's entry on exceptions ... yeah. What do you think about
this:
jmp_buf env;
int i
= 0;
if (
setjmp( env ) <=100 )
{
i++;
printf(
"%d\r\n", i );
longjmp(
env, i );
}
Newsflash: that's
a loop. Using ABBA's
words, I wonder should laugh
or cry...
-
So I have been thinking about C#
lately. C# was born together with .Net and it is commonly thought of as a language
that - having little legacy of its own - exposes most of CLR capabilities in a safe
and consistent way. Sure, Managed C++ extensions let you do a bit more with IJW and
such - but then again, IJW was specifically designed for MC++ and is there to make
migration easier.
Overall the perception seems to
be that C# is the "clean slate" .Net language - lean, clean and gives you all you
need... Most samples you can find out there are in C#, and I have heard plenty of
stories involving companies that switched from VB or J++/Java directly to C# despite
the fact that VB.Net and JSharp.Net are available.
This is probably for the best -
as I was saying before, C# has no legacy of its own so it is thought to have been
designed so that no ugly tricks would be needed to translate language contracts into
CLR execution primitives. That's probably true.
What is also true, is that things
are not quite that simple. There are things that CLR can do - safe, non-ugly things
- that C #doesn't expose. There are things
that C# exposes, which CLR is completely unaware of. And it just so happened that
lately I have come across a bunch of examples of those things.
So I decided to start as series
of posts that look into those things - C# syntax, constructs and capabilities and
what they really translate into as far
as CLR is concerned.
I'll probably start with simple
stuff - like method arguments and delegates - and then just see where that goes.
My methodology is going to be pretty
simple - in most cases I'll be writing some C# code, compiling it and "cracking" it
open with ILDASM (IL Disassembler that ships with Microsoft.Net SDK) and see what
things will translate into... and then speculate on the findings of course :) I definitely
don't expect that people reading this will have any deep intimate knowledge of IL
- I don't have that myself - but I'll do my best to make the results self-explanatory.
All my findings are based on a
particular compiler version - namely one shipped with CLR 1.1 v1.1.4322 (also ships
with Vs.Net 2003). While I believe that most things will probably stay they same in
future versions as well, that - of course - may not be true.
Stay tuned...
-
Intro
entries are for blogs are what "Hello world" applications are for most books - you
need to have one, they all look alike - although everybody wants theirs to look different
- and everyone (including the readers) are very happy to be over with them and move
to the real stuff.
So
this is mine... and of course it's special :)
I'm
a developer on SQL Server "
Yukon
" team - more specifically Business Intelligence (BI) Unit. We are working on quite
a bit of very new and exciting stuff, including Analysis Services, DTS and Reporting
Services ... most of which I can't talk about yet :) ... but I will as soon as I can.
I
have been with Microsoft for 4 years - in fact, exactly 4 years on Oct 18th.
It's been one of the most professionally fulfilling experiences I have ever had, but
this is not what I intend to write about here. You see, I'm your typical geek - I
get a kick out of strange and interesting technical stuff. Also - for some strange
reason - I find it really exciting to know how things really work
and what can be done to make different things
work together.
I don't know why - I just do. And the reality is, sometimes this information is quite
difficult to come across, but that's what this makes it all even more exciting.
I'm
a very curious person and every time I start doing something I tend to try to get
to the bottom of things, which at times turns into a obsessive-compulsive search for
the inner peace ("why does it do that? why??? Must.
Find. Out.")...
and that's not always very productive( shh... don't tell my managers ). At times these
campaigns bring some unexpected discoveries or what I consider interesting tidbits.
And this is what this blog is for me - an outlet for my geeky compulsion :=)
OK,
seriously. Integrating with new technologies is tough. Things are moving pretty fast.
And it doesn't really help you much if you happen to be employed by Microsoft. Well...
it does help, just a little, but not very much.
You see, Microsoft is a very large company, which may look like a huge monolith to
the outside world, but the reality is that internally we are facing exactly the same
issues our customers see externally - many teams, may moving parts, new things come
about every day.
"
Yukon
" team started using CLR, C# and VS.Net long before they became publicly available
- in fact I recall that C# was referred to as "Cool" back then. It's been a pretty
wild ride, and I guess now I feel compelled to share some of that experience with
you. And this what this blog is also about.
While
I'll be rambling about a lot of stuff, mostly I'll be talking about technologies I
have been working with for the last 3 years: CLR
in general and integration with VS.Net(The latter is in fact just un umbrella
topic, it includes Office-based
VS.Net AddIns, a very
little known "Designer Framework" owned by WinForms team (this one is pretty mysterious
- all interfaces are fully public and some topics are selectively covered in a bunch
of articles, but the whole picture is somehow not out there, which is a shame, because
it is really cool) and the very hard-core low-level API behind it all - VSIP
(Visual Studio Industry Partner Program), which went public several months
ago). Now I have been using all this stuff from managed code, which - of course -
means that I will have to talk quite a bit about COM Interop as well. There will be
some stuff on SQL Server/Business Intelligence too ... let's just see how it all comes
along.
It
is important to note that I do not work within Developer Division (which includes
VS.Net and CLR), which means I'm basically in a similar situation as just about everybody
else. Sort of an insider look from the outside, if you will (or is it outsider looks
from the inside?). This means that while I can speculate why things
are implemented in a certain way - and I will - you'll have to take that as just that
- my judgment. What I am always really interested
in is how things
work... and how to make them work they way *I* want
them to.
OK,
this entry is getting waaaay longer than I wanted it to be, so I'll just wrap up.
I should say that if you should definitely check out other GotDotNet
blogs If you are interested
in CLR and Interop, I would definitely recommend reading Chris
Brumme and Adam
Nathan on a regular basis - you'll be getting as much technical information
as you can handle... and probably more :) If you are interested in VS.Net extensibility,
the you also should check out Craig
Skibo's
blog - you'll know much more about the seemingly straight-forward EnvDTE...
which is really not all that simple. :)
So...
the "Hello World" entry is over... Let the "real" part of this blog begin.