Why doesn't the C# compiler accept /resource:*.resources?

grantri — Mon, 19 Jul 2004 20:39:00 GMT

Well, here I am finally getting back to this.

In my previous post on resources, I glossed over one issue. Not all managed resources are exactly what they seem. Specifically at the metadata/compiler level, there is a table that simply points to the resource and gives it a name. This is very similar to how Win32 native resources work. A lot of managed code assumes that data stored there is a .resources file. There is actually a magic cookie value that is at the very beginning of the header for .resources files, but that's not exactly a guarantee that what follows really is a .resources file. That is why the compiler and ALink allow you to embed anything you want as a resource. Likewise Assembly.GetManifestResourceStream, returns just a stream. You have to know what data is in the stream.

As a side note, given this architecture, it seems like the metadata for 'managed' resources adds no value over the existing native Win32 resource format. So it seems like another case of NIH.

Way back before we actually had a good idea of what an assembly was (and before we had any tools that created them) the frameworks team just used raw .resources files sitting on the disk in the application directory. When they moved over to real assemblies, they decided to keep the same naming scheme. This led the compiler team (well basically just me) to come up with the novel idea of using the resource's filename as the default name for the resource. That way they didn't have to type the filename twice.

The problem here is that we weren't thinking about the big picture and how this would look to an end user who hadn't seen the intermediate steps. As a convention most people name their source files according to the class inside it. Thus it would make sense to name a .resource file after the resources inside it, but that doesn't mean we should default it. In fact if I were to do it all over again I would make 2 big changes:

Never default the name, always make it explicit. However, place the name of the resources somehow in the .resources and .resx file itself, rather than on the command-line. Then the designers could do proper code-spit without relying on things like the default namespace or the hidden correlation between the class name and the .resources name. Also we wouldn't have to have a long ugly command-line option to specify the name.
Get rid of the metadata for managed resources, and just use native Win32 resources. Yes the .resources file would still be embedded in the .rsrc section, but we wouldn't need all new tools and APIs to deal with managed resources versus native resources. You also wouldn't have ugly discrepancies like when you set the icon for your main form, but forget to set it as the application icon (which gets it into the Win32 resources, which is where the OS/shell looks).

I still think it would be possible to have managed resource embedded into the .rsrc section, but since they store things in different formats (like icons) it wouldn't help much.

Basically given the fact that I wanted resource names to be explicit, thats the main reason I pushed for not allowing wildcards. There was also the confusion about whether wildcards should be applied recursively. So I guess the end result was that nobody foresaw a huge customer demand, and the few people that cared (like me) seemed moderately opposed.

--Grant

Resources, Managed, Native, and ALink

grantri — Tue, 06 Apr 2004 02:39:00 GMT

So most of you have figured out by now that making your application run with different languages can be very hard. The overall design is always the same: place all content that might change for different languages in some sort of runtime-selectable package, and then load everything dynamically rather than just hard-coding in constants. Now before C# and the CLR, classic windows used a special section in the PE file to do this. The “.rsrc” section was the designated dumping ground for such things. It is designed to be extensible, and localizable. There are 2 obvious and common things in just about any native resource section: the application icon and the version resource. When you use windows explorer to browse through a bunch of .DLLs or .EXEs, the shell cracks open the resource section and looks for the application icon. It it's there it uses it as the icon for the file, otherwise it uses one of it's built-in icons. If you've got a really slow or old machine you can see this in action, all the files will start with default icons and then one-by-one get updated to the right one. The version resource is a little bit trickier. If you right-click on the file and then choose properties, if the file has a version resource, there will be a version tab that shows it's values. Most setup/install applications also look for this resource so they can make sure to upgrade files to the latest-and-greatest. The ugly part comes in because of how this resource is actually stored. Specifically the file version is stored as both a number (2 sequential DWORDs usually represented as 4 unsigned 16-bit numbers) and as a string. Nothing requires them to match, and some tools specifically use the string to contain extra information (like build time stamp, machine, user, etc.). Native resources usually start life as an .RC file, then get compiled to a .RES file using rc.exe, then get converted to a .OBJ file using cvtres.exe, and finally get linked into your native application just like every other .OBJ file. Enough of the native resources, they're all obsolete anyway now that we're in the managed world, right? Wrong, but I'll get back to that later.

Managed resources are a completely new and different beast. For most of you (hopefully) you never really have to know how this works because the designer and IDE project system take care of this for you magically (at least until the magic wears out and starts smelling more like a bug or a hack). Basically managed resources come in 2 file formats: .RESX and .RESOURCES. Neither of them share anything with native resource except the same end purpose. The designer allows you to edit the .RESX file graphically. Then at build time the IDE does it magic and invokes resgen to compile the .RESX file into a .resources file. Finally the C# (or VB or C++) compiler embeds (or links) the file into the final output. The .RESOURCES (and the .RESX) file itself is very structured with strong type information for everything stored in it. There is no chance for accidentally loading an icon as a string, just like the native resources. However, at the metadata level, the managed resources are just an arbitrary blob in the assembly. They come in 2 flavors: embedded and linked. Linked means, “some other stand-alone file in this assembly” and there is metadata to indicate which file, and that whole file is considered the 'resource'. Embedded means, “the resource is stored in this file at offset XXX”. At that offset you will find a 32-bit unsigned integer that gives the size of the resource, which starts immediately after that. Notice the complete lack of types? To the CLR resources are just arbitrary blobs that identified by strings. Now the frameworks and designers have tried to enforce a convention on top of that. Specifically, they assume that the resource names are:

Not case sensitive.
Correspond to <typename>.<locale>.resources naming convention where typename is the fully-qualified typename that will be consuming the resources (usually a winforms class).
Always be the structured .RESOURCES file format.

As for #1, they generally do a case sensitive lookup since it is faster. Then only if that fails will they do a case-insensitive lookup and report some vague exception if they find duplicates that differ only by casing. So as long as the casing matches , you may never know you have a 'problem'. This is especially problematic because several tools rely on the compiler's default which is to use the resource filename as the resource name. (If I could roll back time, I would have never allowed such an awful default). Since some file systems preserve casing and some don't, this could result in different resource names, just because the file system changed!

As for #2 this is generally where the IDE's magic falls apart. Rather than the designer just spitting out the name it used, and then resgen reading it out of the .RESX file (or any number of other sensible options), the project system has to sniff your sources and guess what name the designer used. It does this by using the “Default Namespace” project property setting (which doesn't always match the actual namespace used int eh source file), and then assumes that everybody only puts on class in each source file, and thus uses the first class in the source file. It combines that information to produce the proper .RESOURCES filename which is then used as the string identifier in metadata because they don't bother to override the default.

As for #3, they actually do check! There is a special magic number at the start of .RESOURCES files that they use to validate them. Although I personally think it would be better if this were somehow encoded in metadata...

Now how does ALink fit into this, and what about native resources?

Well not having a version resource would break most, if not all, installers who check for this information to determine if it's safe to upgrade or not. Not to mention the IT experts and QA teams who have grown accustomed to using this information when comparing files. We could have written new tools and APIs that would just gather the information out of the metadata, but that just wasn't back-wards compatible enough for the powers that be. Instead they decided that all managed code should be forced to have a version resource that duplicates the same information into the old native resource section to better interop with old code. Since ALink was already the common-dumping ground, it got this task. Basically it just tracks all of the assembly metadata, and then produces a .RES file that contains a native version resource and optionally an application icon that the user specifies. If you don't like the default version resource the command-line compiler allows you to supply your own, but if you do that, then you have to make sure the version resource in there matches the assembly version info.

Now there are a few really advanced people who realize that adding a whole section to a PE file just to get a version number is sometimes a fair bit of waste, especially on the compact frameworks. Well since there's no way to turn off the auto-generation of the version resource, it might look like you're stuck, but I have found a work-around. By creating an ultra small .OBJ file that has effectively an empty .RSRC section it fools the compiler into not emitting any .RSRC section! So here the .OBJ file: http://mysite.verizon.net/grantri/empty.obj

--Grant

GrantRi's WebLog [MS] : Resources

Why doesn't the C# compiler accept /resource:*.resources?

Resources, Managed, Native, and ALink