PE resources must be 4-byte aligned, but that doesn't stop people from trying other alignments

re: PE resources must be 4-byte aligned, but that doesn't stop people from trying other alignments

Steve Wolf — Mon, 13 Jun 2011 14:41:05 GMT

There are an awful lot of comments that boil down to:

But I don't understand why hardware isn't an idealized perfection without any trade-off compromises between infinite ease of use and infinite speed?

And then they go on to whine that this is intolerable, and their ignorance proves that the hardware guys have it all wrong, and that they shouldn't have to learn anything or think more deeply than their idealized projection of what should be.

And I assume your code is this perfect, has never made trade-offs between any opposed and intrinsically exclusive possibilities in the real (as in reality) world?

Go get and education. Learn why alignment is an issue at the hardware level. Become a more worthy programmer who understands his tools more deeply, instead of whining that they're imperfect*.

*BTW, news flash: the world is imperfect.

re: PE resources must be 4-byte aligned, but that doesn't stop people from trying other alignments

Reid — Sun, 12 Jun 2011 10:09:23 GMT

@ficedula: You are right. But at the same time, what Pierre B. says, also applies:

The simple law is "An unenforced rule is not a rule."

Works in software, works in real life, works when raising children.

As long as we talk about humans who write software, this is almost generally true.

The final conclusion depends on the very details of the subject at hand.

re: PE resources must be 4-byte aligned, but that doesn't stop people from trying other alignments

ficedula — Sat, 11 Jun 2011 19:23:05 GMT

@Burak: It makes sense because virtually *all* architectures either insist on alignment or impose a speed penalty on unaligned access. So pretty much any "data must be aligned to X byte boundary" requirements in pretty much any structure or format has at the very least a *plausible* explanation.

Also, bearing in mind that some Windows formats date back to the 386 days or earlier, you have the problem that enforcing every single rule (including the ones that are there for performance reasons) under every situation takes memory and time which kind of defeats the point of having them. Sure, the alignment one would be pretty easy to check, but if Windows had verified that from the beginning, Raymond would be posting about how some common non-MS resource compiler instead failed to enforce the other future-proofing rules (some of which would be quite expensive to check at runtime, like anything that involves actually examining the resource data: paging in a bunch of resources that might never actually be required by the program can be horrifically slow).

Net result: Enforcing all the rules is often not possible, assuming sane objectives. The alignment one *could* have been enforced, but the general rule for early versions of Windows was "assume the programmer knows what they're doing, we can't afford to waste time and memory checking/cleaning up after them".

re: PE resources must be 4-byte aligned, but that doesn't stop people from trying other alignments

Neil — Sat, 11 Jun 2011 10:38:05 GMT

@Ivo: Sure, everyone has access to BeginUpdateResource now, but older installer generators that needed to run on platforms that didn't provide it would have had to have a custom resource compiler, and therefore older installers could have had unaligned resources.

re: PE resources must be 4-byte aligned, but that doesn't stop people from trying other alignments

Burak KALAYCI — Sat, 11 Jun 2011 09:32:50 GMT

@ficedula: How can you assume a certain hardware architecture and then say PE spec makes sense because of that unmentioned assumption?

If there will be a performance hit, you make it a recommendation, not a rule. If you make it a rule, you enforce it. If you didn't need to enforce it, it means the rule was not necessary in the first place. If you program using a high level language like C, not to mention any toy scripting languages that use a GC, mentioning a performance hit does not make sense.

re: PE resources must be 4-byte aligned, but that doesn't stop people from trying other alignments

ficedula — Fri, 10 Jun 2011 21:40:02 GMT

@Evan: The topic is specifically talking about *resources* in PE files needing alignment. So generally, we're dealing with a section of a PE file that's been copied (or, more likely, mapped) into memory that contains a number of different resource items; e.g. bitmaps, icons, menus, ...

If you can mandate that each resource is aligned on (say) 32-bit boundaries, you can access that resource data (which probably has its own internal structure; a bitmap header at the start of a bitmap resource...) through a simple pointer and assume that all the structures within the resource in question are correctly aligned (your 32-bit field in the bitmap header is aligned to a 32-bit boundary). So you can access the data through a pointer *without* copying it around: that's the point.

If the resource could be arbitrarily aligned, then to access the data, you either have to copy it to an aligned destination (inefficient), or access every single field in the resource one-byte-at-a-time (unaligned data, so pointers to anything bigger than a byte won't necessarily work). That's quite a pain.

On x86, of course, your pointers *will* still work on unaligned data - but they'll be slower, so there's a benefit to mandating alignment even there (although the consequence of failing to do so is a speed penalty rather than a crash). All things considered, the PE spec requiring alignment on resources makes a lot of sense: even on the CPUs of the time, it made things faster, and as an added benefit paved the way for running on architectures that weren't so friendly to unaligned pointers.

re: PE resources must be 4-byte aligned, but that doesn't stop people from trying other alignments

Ben Voigt — Fri, 10 Jun 2011 19:16:20 GMT

Clearly we need a new acronym for PE files, since "portable, executable file" just isn't accurate.

re: PE resources must be 4-byte aligned, but that doesn't stop people from trying other alignments

Evan — Fri, 10 Jun 2011 16:46:08 GMT

@Burak: "What's next, compilers imposing code formatting rules? (Please don't make me remember)."

Ever used Python? Or to a lesser extent, Haskell?

Here's my take: from a hardware side, alignment restrictions make sense. Allowing misaligned accesses means you've got more hardware to deal with it, and doing it *properly* (i.e. not a leaky abstraction, which it still is; see the performance degradation and atomicity issues with misaligned accesses) would take even more. Spend those transistors on something that actually matters and will help the end user, like a larger cache or something instead.

From the PE side, it probably makes less sense. It's not like PE files require a file-system block alignment so that the system can set up a mapping from a memory page to a block (or more; not sure how big blocks are) of the file. I'm not sure what the 4-byte alignment restriction gains in that case; I don't know what's going on behind the scenes well enough to know for sure. I guess when copying the file contents from one location to another in memory, it allows the system to assume that the source address will be aligned. That doesn't sound quite convincing enough to me if that's it.

re: PE resources must be 4-byte aligned, but that doesn't stop people from trying other alignments

640k — Fri, 10 Jun 2011 16:13:05 GMT

@Burak: 16mb is as useless as 4gb. 64-bit bit addressing is the way to go.

re: PE resources must be 4-byte aligned, but that doesn't stop people from trying other alignments

Yuhong Bao — Fri, 10 Jun 2011 15:57:42 GMT

Daniel Colascione: And unfortunately the Windows kernel itself don't turn the CR0.AM bit on that is required for EFLAGS.AC to work.