Evil Compiler Tricks, and Checking for Pointer Math

re: Evil Compiler Tricks, and Checking for Pointer Math

lally — Sun, 06 Apr 2008 07:13:59 GMT

While already a nice entry point for exploits, how about 16-bit targets? Hitting 2^16 is pretty easy.

[dcl] 16-bit code is always a great place to look when auditing. tends to be easy pickings. You also often run into programmer error because they didn't take into account int promotion.

re: Evil Compiler Tricks, and Checking for Pointer Math

moocat — Sun, 06 Apr 2008 03:06:23 GMT

One question I have is whether that check is ever sufficient for real code.

What I am thinking is that you really need to know that 'len' is not greater then the size of the buffer so you don't read/write some random memory that is not part of the buffer. Furthermore, I neither know of any valid way to get a buffer where the buffer wraps around the end of memory nor do I think there is any reason that could possibly be needed.

So, based on that, if you check that 'len' is less then the size of buffer, the check for wraparound is redundant and unnecessary.

[dcl] that's another way of doing it. you still have to make sure that the multiplication doesn't wrap in either case. This is why I'm ambivalent about fully supporting pointer math within SafeInt - it is more properly done by using another class we have internal to Office, which is BoundedPointer - what it does should be obvious from the name. STL iterators now also solve the problem nicely. There could be cases where it's more efficient to do this first, then other checks later.

[dcl] You could also hit a case where I check to make sure the result is in the buffer, but it actually wrapped, and I'm not at the spot in the buffer I ought to be. So you always have to check both that you got there the right way AND that you're still in the buffer. If either condition fails, you have a problem. It is possible to write checks for some code that will do both at once, and that's obviously desirable.

re: Evil Compiler Tricks, and Checking for Pointer Math

Raymond Chen - MSFT — Sat, 05 Apr 2008 01:41:19 GMT

The gcc optimization is legal because 5.7.5 says "If both the pointer operand and the result point to elements of the same array object, or one past the last element of the array object, [then...]; otherwise, the behavior is undefined." Since the first part is not satisfied (you can't have an array that straddles 0), the second half is operative: The behavior is undefined. According to 1.2.3.12, the standard imposes no requirements on undefined behavior. They even call out ignoring the situation entirely as an acceptable response.

In other words, the language requires you to validate the parameters to the + operator BEFORE you perform the operation, not after.

When adding security bugs to your code is not your fault!

Michael Howard's Web Log — Sat, 05 Apr 2008 01:08:14 GMT

David LeBlanc and I (and a bunch of others) just had a little email exchange about some fascinating integer