The Old New Thing

Why so complicated? Every *unhandled* first chance exception is at least a bug, if not a security vulnerability. In kernel-mode code, even the only-a-bug form is a DoS vulnerability.

I don't think it's bad, necessarily, to have to move FormatMessage bounds checking into user space.  I generally avoid using exceptions to control program flow.  In this case it was probably only a win because

- Over 4096 bytes is a vanishingly unlikely case,

- Back when FormatMessage was actually written, the extra 1 or 2 CPU cycles per loop mattered.

That MSDN article you point to has a bug: it assumes sizeof(TCHAR) == 1 and hence allocates twice the number of pages expected (assuming Unicode).

(and I'm just a lowly PM :-)

"the kernel folks rewrote".

Don't they ever blog anywhere? I haven't found any team or person who writes ntoskrnl.exe and the likes and blogs.

> Why so complicated? Every *unhandled*

> first chance exception is at least a bug

what does this comment mean? This is article is about an exception that is correctly handled.

but people often run debugger, which break on all exceptions, and don't understand what's happening.

@Ulric: even if it doesn't break on all exceptions, you do get a notification in the debug output window that a first-chance exception occurred.

A good thing it was removed. I don't like exceptions being used as logic. They are slow as well.

"They are slow as well."

Not when they don't get called. This particular exception is only called when the resultant string is > 4096 bytes long, which is very unusual.

Personally, I imagine the logic in Vista is probably more complex than it was in XP, just to get rid of the exception.

Cereal: "I don't think it's bad, necessarily, to have to move FormatMessage bounds checking into user space."

Actually it is bad, because you've just crossed a security boundary.  If the kernel doesn't validate parameters passed to it from user mode then the kernel itself is vulnerable to buffer attacks.

While you can mitigate this a bit by having a user-mode wrapper that everyone uses to call into the kernel, there'll still be some evil program that spoofs this wrapper and passes invalid parameters just to ruin the kernel's day.

"Actually it is bad, because you've just crossed a security boundary.  If the kernel doesn't validate parameters passed to it from user mode then the kernel itself is vulnerable to buffer attacks."

Except that, except for SEH (And the VMM if you're pedantic), the kernel isn't involved in any of this. And the SEH handling code hasn't been changed.

lately your posts have been exceptional! i don't know how you keep it up.

>> A good thing it was removed. I don't like exceptions being used as logic.

Except is a one of the most common techniques around in operative systems..

Copy-on-write on *nix systems (and maybe not only) is implemented in the same way, for instance. Think about it the next time you fork.

"...to avoid these sorts of false alarm security vulnerability reports, the kernel folks rewrote the FormatMessage function in Windows Vista so it doesn't use this technique any more...."

Did they take out stack guard pages as well?

@Dean, I doubt "if index >= bufferlength" is overly complex logic.  If there's just one of those, or a small handful, it probably saves code size compared to the SEH setup and teardown.

@Miral, when I say "moving into user space," I just mean it doesn't involve a context switch anymore.  Obviously I'm not advocating that kernel buffer checks (those which are done in software in ring 0) should be moved into user space.

/* Using intentionally invalid pages to trigger just-in-time memory commit is a rare technique,

*/

Interestingly, last Friday I was reading up on Steel Bank Common Lisp, and they do something on Unix via intentional segfaults ( http://sbcl-internals.cliki.net/Signal%20Handling "Some signals are used by the Runtime. ... [T]ypically this includes ... Whatever you get when writing into unmapped memory (SIGSEGV, or SIGBUS) - used by the GC for triggering a collection." ).

> Personally, I imagine the logic in Vista is probably more complex than it was in XP, just to get rid of the exception.

No, it's actually simpler and more efficient to pre-commit the entire buffer (which is what Vista does).

Commiting on demand is only necessary if the combined size of the buffers is so large that you need to worry about the system commit limit (thread stacks is a good example). Otherwise, just commit everything in one shot. A committed page that is never accessed doesn't use any physical memory, so it is just as efficient as a reserved page.

"which is what Vista does"

Really? That's how it does it? How can it know the "combined size of the buffers" without parsing the format string first (for example, an insertion point may be repeated many times -- for example, "Hello %1 World %1").

The "combined size" comment was referring to the general technique of committing memory on demand (as in the MSDN sample Raymond linked to above).

I don't know how FormatMessage decides what size to allocate. That part probably didn't change in Vista. The main thing that changed is that now instead of reserving a region and using try/except to commit individual pages, it commits the entire region at once.

Oh, I get it, thanks :-) That's what I get for trying to guess the implementation! I suppose it could do two passes over the format string: once to find out the final size that it'll need, and once more to do the actual processing.

"Using intentionally invalid pages to trigger just-in-time memory commit is a rare technique"

Probably because it's a pain to do right and not that many programmers know (or care) how to do it.