David LeBlanc's Web Log : Integer Overflows

SafeInt Compiles on gcc!

david_leblanc — Wed, 26 Nov 2008 06:59:00 GMT

[update 12-1-08] I now have it completely compiling on gcc, with a test harness that exercises every method of the class for every combination of types (all 15 of them). Version 3.0.12p is now moved to release status.

Once I got SafeInt posted on CodePlex, Niels Dekker grabbed a copy and started figuring out what needed to be done in order to get SafeInt to compile using gcc. Niels was one of the first people to give me bug reports during the development of SafeInt 1.0.x, and he's been extremely helpful over the years – hard to believe I've been working on SafeInt for 5 years now. When we first tried to compile it with gcc, gcc didn't have the template support that SafeInt needed – it was a complete mess, and we gave up quickly – about as bad as trying to get it to compile using Visual Studio 6.

As it turns out, I'm also working on updating the "19 Deadly Sins of Software Security" – there's going to be 2 dozen sins this time around, and I set up a Linux system so I can test things. I originally started running Linux in 1993 – I think it was version 1.0.3, and have tried to keep one running for most of the time since. Having a Linux system handy, I also had g++ available, and I could then actually test some of the issues Niels pointed out – and found a couple more – seems that gcc on cygwin != gcc on Linux. It took about 3-4 hours, but since Niels had already pointed me in the right direction, it went much more quickly than if I'd had to puzzle it out completely by myself. Seems that gcc is a little stricter about some things than Visual Studio, and some of this is a very good thing.

Here's an example of an insidious issue I'd never given much thought to – say you had some function with the following signature:

void Foo(unsigned int& cch);

If you passed it some argument, you would expect it to get modified, right? That's what references are for. Now let's say you called it like so:

unsigned int x;
Foo((unsigned int)x);

Would the cast change anything? How about if we did this?

void Foo(bar_class& cch);

bar_class x;
Foo((bar_class)x);

As it turns out, the cast creates a temporary copy, and the temporary copy _might_ be the one that gets modified. The Visual Studio compiler will do the right thing if you're dealing with a native type – like unsigned int – and go ahead and discard the cast and modify the argument as you'd expect. Unfortunately, if it is not a simple type (haven't tested with things like simple structs), it is the temporary copy that gets modified, and the argument does not. The most recent released Visual Studio compiler doesn't even complain about this >8-O OTOH, the gcc compiler does, and while it wasn't a runtime bug in SafeInt, it was non-standard C++, and it's now fixed in the 3.0.12p line.

A gcc issue that I think is not a bad thing for Visual Studio is in the area of implied template parameters – say you did this:

template <typename T, typename U> class Foo {
template <typename T> void FooMethod(){ Bar:: foo<U>();}
};

You'd expect the U parameter to be the same as the enclosing class, and with the Visual Studio compiler it is – but gcc complains. In order to fix it, you have to use 'template' keywords, like so:

template <typename T> void FooMethod(){ Bar::template foo<U>();}

Despite Niels pointing me to a helpful page, I'm still not sure why the template keyword is really anything helpful. Maybe someone reading this can illuminate me on the topic.

Some of you may be wondering why I care about SafeInt on gcc – after all, I do only really write code for Windows, and have for a long time. The first reason I care about this is that I'd like it if SafeInt were something that could be used by people writing cross-platform code – a lot of our customers do this, and if I can eliminate one more reason to have to fork code (or not use SafeInt), that would be nice. The second reason is that we do have quite a few people developing for MacOS here – MacOffice is the biggest example, and Apple for some reason requires gcc to compile. It would be really nice if we could use SafeInt on all of our code, not just Windows code.

If you'd like to check out a version of SafeInt that does compile on gcc, you can get it here - http://www.codeplex.com/SafeInt/Release/ProjectReleases.aspx?ReleaseId=19785

I've also included the start of a public test harness for the class, which isn't complete just yet – which is why this release is marked planned, not released.

Ptrdiff_t is evil

david_leblanc — Wed, 03 Sep 2008 02:15:00 GMT

Well, not really, but here's a code problem that confounded some really smart devs – and it looks so simple!

void IncPtr( unsigned int cElements )

{

if( m_pMax - m_pCurrent > cElements )

m_pCurrent += cElements;

else

throw;

}

OK, so here's the question – if an error has happened, and m_pCurrent is > m_pMax, which implies the difference in the pointers is negative, which code branch do we execute? Assume cElements is a reasonably small number.

Hmmm – the immediate answer would be that the difference gets cast to an unsigned int to be compared with cElements, if it is negative, then it becomes large, and it is not less than cElements, so we throw, so this code is safe, right?

The answer, unfortunately, is a solid maybe. Back in engineering school, I got acquainted with something called dimensional analysis where you worked something out based on dimensions. For example, if you want to know how to get gallons from some number of miles and miles/gallon, figuring (miles / (miles/gallon)) shows the answer is in gallons. A similar approach for integers is often helpful. Let's look at the types involved in the if statement. First, what is the type of a pointer difference? That's a ptrdiff_t – which is a signed number that is the same number of bytes as a pointer, which means that on a 64-bit build, it is an __int64, and on a 32-bit build, it is a 32-bit int.

What we now have is:

If( ptrdiff_t < unsigned int)

If you have a 32-bit build, this works out to:

If( int < unsigned int)

Which then implies:

If((unsigned int)int < unsigned int)

The cast gives you an implied check for the lhs being less than zero (assuming reasonably small values for the unsigned number), and negative numbers will now fail, and since this is an error, this is what we want, and life is good. Do note that if you're compiling with all the warnings on, this will cause a warning, which you'd probably ignore, or cast away, being oblivious to the impending doom that is approaching.

Now consider 64-bit:

If(__int64 < unsigned int)

This gets cast very differently…

If( __int64 < (__int64)unsigned int)

You won't get a warning on your 64-bit build because the cast from unsigned int (like the cast from unsigned short to int) preserves value, and the assumption is that the comparison will always be correct. Under the error condition outlined here, the problem is that we're now not catching the error, we'll add to a pointer that's already probably bogus, and things will get worse from here.

As you can see, which branch gets executed depends on whether you're building 32 or 64-bit!

The solutions and lessons are –

Be wary of pointer math in porting code to 64-bit – ptrdiff_t is negative, and changes size.
Using the bit flipping of negative to very large positive effect is really programming with side effects, and being clever, neither of which are good practices.
Explicitly determine which path to take when dealing with negative numbers in a comparison

BTW, SafeInt will be available very shortly on CodePlex – I have just one more internal hoop to jump before I can post it.

More on Checking Allocations

david_leblanc — Tue, 22 Apr 2008 05:27:00 GMT

Seems my last post met with some objections – somewhat rightfully so, as I mischaracterized one of Tom's points – he never advocated just not checking for allocations, but instead to use an allocator that has a non-returning error handler – though it seems some of his commentors were advocating that (I think they should go to rehab). This isn't always a bad way to go – I recommend using SafeInt the same way in some conditions. There are some problems that make allocators special, though. For example, say I write a DLL that allocates internally. If the DLL just calls exit() when it can't allocate, this could cause some really bad experiences for users. Same thing if the DLL just tosses some exception that the client isn't expecting. The right thing for such code to do is to either trap the error, and propagate it back up the stack, or be exception-safe code, and throw an exception that is caught just prior to returning to the client code, and throw an error there.

But wait – what about client code? I've said that if you get some weird input that results in bad math going into an allocator, then it _might_ be better to crash than to run arbitrary shell code, but the user experience still stinks. You've just improved things from disastrous to merely bad. Let's look at a concrete example – PowerPoint is all really nice exception-safe C++ code, and they're hard core about doing it right. Inside PowerPoint, allocators throw, and they handle errors where they make sense – typically by unwinding all the way back to where you can cleanly re-start the app, and not blow away other presentations you might be working on. PowerPoint, along with most of the rest of Office calls into mso.dll, which is mostly very much not exception safe. If it started throwing exceptions into say Excel, this would not be a good thing. Thus, that code has to check every allocation, properly clean up, and return an error.

The real kicker to all of this is that we're entering a really interesting 64-bit world. My current motherboard handles 8GB, and if past trends hold, in the next 10 years, I could have over 100GB RAM in my system. The software we write today could well be in use for 10 years. Some whiz-bang 3-d video-intensive app could very easily think that allocating 6GB was a perfectly fine thing to do, and if crashes on one system and runs on another, that's not a happy user experience. In the 64-bit world, we could have an allocation size that's absolutely correctly calculated, doesn't represent an attack, and it could quite easily fail on one system and not another. This isn't the sort of thing we want to crash over. I've got an implementation of vi written around 1993 that just exits if the file is bigger than about 1MB – seems really silly now. I do have a game installed at home that had a bug in it because some library got upset when it tried to use > 2GB for a video buffer. Stuff like that is going to be annoying when I build my next system in a few years and have 32GB or so.

Another issue is that crashes are really serious exploits to server code, even if the service restarts – the perf implications are horrible. I've seen instances where someone wanted to run what was considered client-side code on a server, and trying to get it to server level of quality was tough. You usually don't know when this might happen, so I'm kind of hard core about it – write solid code, and pick your poison – either don't use exceptions and check ALL your errors, or if you have the luxury of dealing with exception-safe C++ code, then do the work to handle exceptions in the right places, and use throwing new. Note that server code has been moving to 64-bit for some time.

Yet another reason to check these things, at least in existing code, is that we do often correctly handle out of memory conditions locally – "No, you cannot paste a 2GB image into this document, try something smaller" being one example. If I then add a check for int overflows in the same function, I don't want to in general introduce new error paths. What I can often do is make the int overflow show up as a bad alloc. For example:

Template<typename T>
size_t AllocSize(size_t elements)
{
if( elements > SIZET_MAX/sizeof(T) )
return ~(size_t)0; // can't ever be allocated

return elements * sizeof(T);
}

It then becomes really easy to guard against int overflows in that code base.

Something else that jogged my memory – I wrote here about a great presentation by Neel Mehta, John McDonald and Mark Dowd on finding exploits in C++ code. The thing is that there aren't many developer mistakes that don't lead to exploits one way or another. This is actually independent of language – there's things peculiar to C++ that can result in exploits, stuff that only C# can do, weird tricks with perl, and on and on. If you want to be a better developer, reading the Effective C++ series is a big help. If you want to be a better code auditor/tester, learn to be a better developer, and you'll be better at spotting programming flaws.

Checking Allocations & Potential for Int Mayhem

david_leblanc — Thu, 17 Apr 2008 00:37:00 GMT

Must be synchronicity. I started out the day with a really interesting mail from Chris Wysopal talking about how allocations can go wrong, fun with signed int math, and the new[] operator. Once I got done responding to Chris, I then notice Robert Hensing's blog pointing me to Thomas Ptacek's comments about Mark Dowd's new exploit, found at http://www.matasano.com/log/1032/this-new-vulnerability-dowds-inhuman-flash-exploit/.

Some people were even saying that it makes no sense to check returns from malloc. I'm not sure what psychotropic substances were involved in that conclusion, but I don't want any. Let's look at all the ways this stuff can go wrong.

First, let's assume a non-throwing new (or new[]), or a malloc call. If you don't check the return and it fails, then we have roughly the following 3 categories, all of which are bad:

No one adds anything user-controlled to the pointer, and it just gets used directly (or null + small, fixed offset). Your program crashes. If some other vulnerability exists, maybe you execute arbitrary code on the way out. I've written about this sort of thing before here and here. This is bad, customer is not happy, and if it is EVER used on a server, it is very bad.
Someone adds a user-controlled value to the offset (and you didn't validate that, also bad). We now have an arbitrary memory write, and while Mr. Dowd is LOTS better at shell code than I want to be, making an exploit from one of these has the hard part mostly done.
Next, and this is an interesting twist that's obvious once you think about it, is when someone then does this – ptr->array[offset], where offset is user controlled. Having ptr be null could actually even be an advantage, since you have an absolute starting point!

Now let's consider that we take the code and port it to 64-bit. On 32-bit, we can assume a 2GB alloc is going to fail. It has to. It's normal for an allocator to have code along these lines:

if( (ptrdiff_t)size < 0 ) fail_alloc;

Now go and change things to 64-bit, and a 2GB alloc might actually succeed, and there's actually no knowing what the upper bound might really be. You can now no longer say that 2GB ought to be enough for anybody, because it might not. Thus, in the non-throwing case, not checking returns is asking to get hacked, especially on 64-bit.

Next, let's consider the throwing case. This is something I was talking about with Richard van Eeden that's an example of bad code. He pointed this out:

Foo* pFoo;

try{ pFoo = new Foo;}

catch(…){delete pFoo;}

Obviously, there would be more to it than this, but what happens if new – or just as importantly the Foo constructor – throws, then we delete pFoo, which is uninitialized, and we should ALWAYS consider uninitialized variables to be attacker controlled. The problem here is that the programmer did not recognize the try-catch as a branch, and didn't ensure that pFoo was initialized everywhere. Since this is moderately subtle, it's worth checking for in an audit.

Now on to an interesting wrinkle – what about that throwing constructor? It's my preference not to do this if I can avoid it, but YMMV. So here's something interesting – now say someone wrote this:

class Foo

{

public:

Foo()

{

p1 = new Thing1;

p2 = new Thing2;

}

~Foo()

{

delete[] p1;

delete[] p2;

}

Thing1* p1;

Thing2* p2;

};

If the first allocation succeeds, and the second fails (or could be more complex constructors), then because the constructor never completed, the destructor never runs. This is thus dangerous code (especially if the first thing had global side-effects external to the app). There are two correct ways to fix it. The first is to properly use resource holders, like auto_ptr, and those will properly clean up even if the actual containing class destructor never runs. This is nicer, and you should _always_ use these when working with exceptions. The second way to make it work is to not do anything that can fail in the constructor, mark it throw(), and provide an init() method. Now the destructor has to run, things get cleaned up, even if init only makes it ½ way.

This leads me to my second main point, which is that if you're using throwing allocators, then you MUST write exception-safe code. If you do not, then something is going to get in a bad state when the stack unwinds, and it's quite often exploitable.

I hope I've established pretty firmly that not checking allocation returns is a very bad thing, and that not using proper exception safe code when throwing exceptions (assuming that you're not using the exception as a proxy for CatchFireAndBurn() ) is also a prescription for disaster.

Lastly, something else that's an interesting side-effect of the fact that the VS 2005 (and later) new[] implementation detects int overflows, we can actually use it in some respects to check math. So let's say I pass an int into new[]. First problem is that if the int is negative, the allocation is going to fail as being too large, even if sizeof(element) == 1. If it succeeds, we know the value is >= 0. Next, count * sizeof(element) has to not overflow. For this to be true, count <= INT_MAX/sizeof(element), also has to be true, and we may then be able to leverage that to avoid having to do redundant checks on count further along in the function IFF we checked our returns.

More Checking for Pointer Math

david_leblanc — Wed, 09 Apr 2008 04:13:00 GMT

Someone pointed out that it isn't sufficient to check for whether the pointer math wrapped, but that we also need to check that the resulting pointer is in our buffer. They then came to the possibly erroneous conclusion that really all you had to do was to check whether the resulting index was in range. The real problem with this is that there's so many different scenarios that I don't see a one size fits all technique.

Depending on the code involved, it may not be sufficient to just know you're in the buffer. For example, bitmaps have 2 different ways to determine where to start drawing (scan0). Top left corner is easy – it's 0,0. Bottom left corner is harder – it's pixel 0 of the last line. So if I calculated scan0, and it ended up in some random place in the buffer, and then I started drawing up from there, I'd eventually start drawing my bitmap header and pieces of the heap and stack. This might lead to ways to overcome ASLR or other mischief.

Where the comment is absolutely correct is that if we're just doing a simple de-reference (get the nth element), then all we have to do is determine if n is somewhere in the array, and there's no need to do any complicated pointer math checking. If 0 <= n < max_elements, then we're in great shape. One would hope that this isn't the sort of code where we'd be checking if the pointer wrapped anyway. In other cases, I might want to do something more complicated, like calculate where to access some element given a starting point of a variable length header, and the element is in some structure following that header. To use a contrived example, figure out the pointer to the SID contained in the 5^th ACE of a SECURITY_DESCRIPTOR's DACL – ick. This actually brings up a reasonable example – say someone stored a self-relative security descriptor in a file (Excel does this), and you've found out that passing a "malformed" security descriptor to random Windows APIs resulted in a non-exploitable crash. Now go write code to completely validate a security descriptor and all of the associated sub-structures. You have 4 possible elements following the header, and you have to check not only that each of them start in the buffer, but also that none of them go outside the buffer, and worse yet, that none of them overlap. This is quite tricky, since a DACL is a variable sized struct that contains a variable number of ACEs, which are also variable in size, and the ACEs contain SIDs, which is why they vary in size, and so on. Got to be quite a bit of code. The Windows APIs along the lines of IsValidSecurityDescriptor() are of very limited help in this area.

While there are some simple cases that can be done easily, it is a hard requirement that pointer math must be mathematically correct AND the result makes sense in terms of the buffer you're dealing with. And as we now know, we also have to worry about doing it in such a way that we don't run into undefined behavior according to the language.

Evil Compiler Tricks, and Checking for Pointer Math

david_leblanc — Sat, 05 Apr 2008 00:39:00 GMT

My favorite programming geek hobby being integer overflows, this caught my eye –

"gcc silently discards some wraparound checks" http://www.kb.cert.org/vuls/id/162289

Basically, what it says is that code which looks like this:

============ snip ==============

char *buf;
int len;

gcc will assume that buf+len >= buf.

As a result, code that performs length checks similar to the following:

len = 1<<30;
[...]
if(buf+len < buf) /* length check */
[...perform some manipulation on len...]

are compiled away by these versions of gcc

============ /snip ==============

Apparently, the compiler may be allowed by the C/C++ standard to do this, as pointers wrapping are not defined. However, the standard also says that optimizations may not change externally observable behavior in an application (aside from making it go faster), so I'm not so sure this is completely legal, but I Am Not A Standards Geek (IANASG), so I'll let them fight it out. The CERT advisory says to do this:

if((uintptr_t)buf+len < (uintptr_t)buf)

being alarmed by this problem, I then consulted what I do in SafeInt, which is:

template < typename T, typename U, typename E >

T*& operator +=( T*& lhs, SafeInt< U, E > rhs )

{

// Cast the pointer to a number so we can do arithmetic

SafeInt< uintptr_t, E > ptr_val = reinterpret_cast< uintptr_t >( lhs );

// Check first that rhs is valid for the type of ptrdiff_t

// and that multiplying by sizeof( T ) doesn't overflow a ptrdiff_t

// Next, we need to add 2 SafeInts of different types, so unbox the ptr_diff

// Finally, cast the number back to a pointer of the correct type

lhs = reinterpret_cast< T* >( (uintptr_t)( ptr_val + (ptrdiff_t)( SafeInt< ptrdiff_t, E >( rhs ) * sizeof( T ) ) ) );

return lhs;

}

So let me explain what exactly we're doing here, and why the CERT advice is bad (except where sizeof(T) == 1). First thing is to cast the pointer to an unsigned type that can always hold the bits in a pointer, and store that in a SafeInt. This happily coincides with the fact that unsigned int overflow behavior is actually specified. The second thing to do is to ensure that the offset multiplied by sizeof(T) does not overflow, and store the result in a ptrdiff_t (we could be moving backwards in the array), and finally check whether the addition ends up overflowing.

If you write code that only checks to see if the result of a pointer addition is greater than what you started with, your compiler might just remove the check, but what is much worse is that your code might not be working even if the compiler does not remove it. To write really correct code, you have to check BOTH the multiplication and the addition operations. As an aside, a problem I like to give to people as a puzzle is to write the correct code to determine if a + b * c yields a valid result or not. Surprisingly few people get it, but I think we can see here just how often we need to get exactly this operation right. Here's a trick that works for 32-bit unsigned code:

Unsigned __int64 result = (unsigned __int64)a + (unsigned __int64)b * (unsigned __int64)c;

If( (unsigned __int32)(result >> 32) )
return error;

MulDiv Mayhem

david_leblanc — Fri, 08 Feb 2008 00:09:00 GMT

Here's another episode in my ongoing quest to stamp out integer overflows. MulDiv is a Windows API that was around before we had 64-bit integers as native types. MulDiv is defined like so:

int MulDiv(int a, int b, int c)

Ironically, the problem it's trying to get around is integer overflows. If you've done any work with numerical analysis, the problem is obvious – what we're trying to do is (a*b)/c. If we multiply a*b as 32-bit numbers, it could overflow, and if we then divide, we'll get junk. If we did it as a*(b/c), then if b < c, the result is 0 (for positive values of b, c). Speaking of which, I think numerical analysis ought to be required for all programmers, and it helps when what you're working on matters – use bad math to figure out the specs for an airfoil, or just about any engineering design issue, and people could get hurt.

The way to fix this is to do what MulDiv does, which is essentially:

__int64 ret = ((__int64)a * (__int64)b)/c;

if(ret > INT_MAX || ret < INT_MIN)

return -1;

return (int)ret;

This does fix most of the problems, and I use some of this in SafeInt – all possible values of a*b can be represented by a 64-bit int, and the division will take place afterwards, guaranteeing a numerically correct result, if the return fits into an int. Herein lies the first problem observed in real usage – people don't check the return, and worse yet, the error return is a perfectly valid value. Where the real problems come in is when people start passing unsigned ints into this function. What you get is the equivalent of doing:

(__int64)(int)(unsigned int)x

If you happen to have a number <= INT_MAX, the upper bit will be 0, we'll sign extend, the bit pattern and the value are both preserved, and things are good. But if we pass say 0x80000000 into this, the casts have us end up with 0xffffffff80000000. This could very well overflow internally, and put junk into ret. If you had a function that took an unsigned int, then our cast to __int64 gives you 0x0000000080000000 – very clearly a different number, and interestingly enough, you could write UMulDiv as:

__int64 ret = ((__int64)a * (__int64)b)/c;

if((unsigned __int64)ret > UINT_MAX) // could also be written as if((unsigned int)(ret >> 32)), which may be slightly more efficient

return -1;

return (unsigned int)ret;

If you look at what's really going on, a 64-bit int multiplication can be broken up into this:

(+/-1)*(a*2^32 + b) * (+/-1)*(c*2^32 + d)

This now expands to:

(+/-1)*(ac*2^64 + ad*2^32 + bc*2^32 + bd)

Failures happen when ac != 0, ad >= 2^32, bc >= 2^32, or if the result of (ad+bc)*2^32 + bd overflows on addition. In the case of a large unsigned value coming in, we'll be very likely to fail in the second condition (ad or bc >= 2^32), since 2^31 times anything bigger than 1 meets our failure condition. So if you tried to calculate 2^31*2/4, and expected to get 2^30, you'd get a failure with MulDiv, and in fact, it does give you the wrong answer, with respect to an unsigned input – though it is correct with respect to signed math, since abs(0xc0000000) = 0x40000000.

int main(void)
{
printf("%x\n", MulDiv(0x80000000, 2, 4));
printf("%x\n", (int)((__int64)(0x80000000)* 2/ 4));
}

Yields:

C:\scratch>muldiv

c0000000

40000000

I should go look up the implementation, since if you feed it INT_MIN as every argument, it should return INT_MIN (the second line does), but instead it fails with -1. I'm not sure why that would happen, but since dividing by INT_MIN is a corner case, I'm not surprised.

You still obviously have the problem of a valid result being returned as an error, but essentially, passing unsigned values into MulDiv is a prescription for mayhem, and assigning MulDiv to unsigned numbers is just as bad. Another problem I've recently run into is this:

short s;

int i = MulDiv(s, 5, 6);

This is really just a silly waste of cycles – the operator casting does everything just fine, and what you end up with is the equivalent of:

int i = (int)s*(int)5/(int)6;

This is going to work just fine, especially if the fixed multiplier is smaller than the denominator – though again, you could hit truncation problems if you assign the results back to a short and we're making the value larger.

How to fix it? We could write a fairly large set of overloaded functions that take every combination of signed and unsigned input types, but if you want both signed and unsigned returns, you'll end up with 64 functions, and that's if we stick to the 32-bit inputs, and if you're overloading, don't forget to duplicate the ones for int and long (or ask your users to cast). A more correct implementation in C could be done simply by returning an error code, and putting the result into a pointer – we do have two failure modes – overflow and divide by zero. Or we can do this easily with SafeInt:

MY_INT_TYPE m = (SafeInt<__int64>(a)*b)/c; // note – parens not needed – for clarity only.

It will just do the right thing for nearly any combination of types, excepting if a and b are themselves 64-bit – fixing that one is hard, and you'll end up doing the multiplication and division by hand, register by register. Speaking of SafeInt, we're working on some changes that will tidy some things up and allow us to put it on MSDN again. Stay tuned.

Now that we do have 64-bit ints, it is almost always going to be better to just do this inline and do whatever checking makes sense for your code – and if you're scanning code for int overflows, doing a grep on MulDiv will very likely find some problems. I'm starting to think that putting MulDiv on the banned API list might be a good idea.

Unsafe String Handling with strncpy

david_leblanc — Wed, 06 Feb 2008 02:20:00 GMT

I recently ran into a piece of code that looked like this:

int len = cchIn;

strncpy(dest, src, len - 1);

This is bad, because strncpy is defined as so:

char *strncpy( char *strDest, const char *strSource, size_t count );

The original complaint was that we were passing a signed int into a function that takes an unsigned int, and while this is a bad thing, it won't really save you. In fact, if we had this:

unsigned int len = cchIn;

strncpy(dest, src, len - 1);

It's just as vulnerable as the previous version, and perhaps worse because the code analysis tool might not call our attention to it. What happens if we somehow make len = 0? 0 – 1, whether signed or unsigned, still passes 0xffffffff to strncpy, which is really the same thing as just calling strcpy! Oops.

This illustrates why strncpy_s has a different signature –

errno_t strncpy_s( char *strDest, size_t sizeInBytes, const char *strSource, size_t count );

Even if you foul up count, you're unlikely to foul up sizeInBytes, which would then catch the error.

Lessons learned – while it is always better to use the type the function wanted in the first place, and you should always be careful of signed-unsigned mismatches, just using unsigned numbers might not save you. Second lesson – doing math inside a function argument where all sorts of bad things can happen and you can't check anything isn't wise. Last and most important lesson is that there are good reasons why we now have strncpy_s and friends, and just because you're not using strcpy doesn't mean your code is safe.

Personally, I prefer to use string classes to prevent a lot of this sort of mayhem. When IIS 6 was being developed, this was one of the major changes they made to prevent buffer overruns. Given IIS 6's really good security record, I'd say they have some strong arguments for following their example. We make use of a lot of these in Office, too.

Templatized Min/Max Solved!

david_leblanc — Thu, 31 Jan 2008 05:42:00 GMT

I had some time to think about the overall problem, and had originally thought of a functional approach, like so:

template <typename R, typename T, typename U> R Max(T t, U u);

This has all the information we need to check for truncation on return, but it requires that the programmer know what the return type needs to be. This seems to be a modest constraint, but in practice, people foul this up, and it's subject to problems where we'd like to just pass the result directly into a function, and what if the function changes signature?

So let's drop back and look at the requirements:

The function must deal correctly with mixed types (e.g., (DWORD)2 > (int)-1)
We must be able to correctly determine if the return can be properly assigned.
Programmer should get to choose what happens if the return cannot be properly assigned (maybe returning the max for that type makes sense?)
We should not evaluate anything more than once.

The first constraint is easy. There are exactly 5 cases for correctly doing a comparison – no cast needed, cast to int, cast to int64, and 2 hard cases involving unsigned __int64 mixed with signed ints that needs a little special logic. I have this code written in SafeInt already, so let's re-use that. Here's the start:

template <typename T, typename U>

class SafeMax

{

public:

SafeMax(T t, U u) : m_t(t), m_u(u){}

private:

SafeInt<T> m_t;

SafeInt<U> m_u;

};

We now have a class where forcing the assignment to a SafeInt keeps you from fouling things up because say classes Foo and Bar both implement the < operator, and you can feed both of these to a typical max macro, and then the results may or may not compile, and likely don't make sense. Putting things into a SafeInt means we'll only work with regular integral types. We might want to allow for some overloads in case a SafeInt gets passed in to start with. This also has the very nice side-effect of meeting requirement 4, which is that we don't evaluate things more than once.

Since we're now working with SafeInt internally, it will go ahead and not only do the comparison correctly, it will pick just the right function to do the job because of partial template specialization, so this deals with the first requirement that our max function not tell us that -1 > 2.

Meeting requirement 2 had me baffled for much the same reasons Andrei Alexandrescu gave in his article about this problem. However, if we make a simplifying assumption that we only need to deal with integral types, which solve at least 99% of our problem, life gets much simpler. In SafeInt, one of the best features is that it will correctly cast itself to whatever a function requires. This is how we can change code flow according to return type! There are 11 of these (remember that char, signed char and unsigned char are 3 different types, though effectively are only two, and while long and int are really the same thing on most platforms, as far as the compiler is concerned, they're both different types), and the implementation is simple – just do this:

operator char() const { return Max<char>(); }

The implementation of Max() is much the same as what we typically use, except that I added logic to check to see if the value being returned would actually fit in the type requested. In my current implementation, I'm going to allow for a #define to change whether it will just return the largest int of that type that actually fits (after firing an assert), or whether it throws an exception like the rest of SafeInt.

Overall, it's astonishingly hard to make a proper C++ rendition of something as simple as min and max, but by leveraging partial template specialization so that our common cases are efficient and by realizing that if you force a class to be unboxed only through the casting operators, you then get to run different code on the basis of the return type. The really nice thing about this is the ease of use – it will appear to the code very much like the macro we're replacing:

int foo = SafeMax( a, b );

We don't have to worry about the types of a and b, nor do we get multiple evaluation warnings, and we don't get signed-unsigned mismatch warnings, either. Even nicer is that say we have some function Bar, which takes some int type as a parameter, we can just do this:

Bar( SafeMax(a, b) );

If you're in fixing code, there's no immediate need to go sort out what type Bar was expecting. Obviously, you'd like to at least be aware if we're taking the max of an int, an unsigned int and assigning the result to a short, but if there are constraints on the inputs, maybe there isn't a problem.

Templatized Min/Max is a bad idea!

david_leblanc — Fri, 25 Jan 2008 08:28:00 GMT

Ah, back to nice geeky C++ programming topics, which is much more fun than angry customer topics…

Some well-meaning soul wrote this:

template<typename T, typename U> T TMax(T t, U u){ return t > u ? t : u; }

Let me count the bugs – first of all, it still runs afoul of the normal problem with max, which is this:

UINT u = 2;
INT i = -1;

TMax(i, u) will return -1, because in order to compare the two, they have to be cast to a UINT, and (UINT)-1 is 0xffffffff, which is clearly a much bigger number than 2, so yeah, the maximum of 2 and -1 must be -1. Sure, that's gonna work…

If you're an astute C++ template reader, you might say, hey – what's the return type there? Good catch. What happens if we do this?

WORD w = 0xffff;
INT oops = 0x10000;

TMax(w, oops)

Well, then it correctly figures out that 0x10000 is bigger than 0xffff, and it will then truncate it back to a WORD, so the maximum of 0xffff and 0x10000 must be 0, right?

I'm sure this could do some amazing things in terms of unexpected code flow, allocations, and so on. You may be wondering how I found this? I was working in a code base that had:

#pragma warning(disable:4244) // Integral type converted to smaller integral type

So I commented it out, rebuilt, and found a complaint we were truncating ints to a WORD.

This is pretty icky, since what we really want is a way to change the return type based on the two inputs. As it turns out, we need signed and unsigned, 32 and 64-bit ints to do the job. I'm not sure it is possible to make this happen very easily without overloading the 121 possible combinations of integer types, along the lines of:

Int TMax(char a, char b);
Int TMax(signed char a, char b);

So how do you fix such a thing? I leveraged SafeInt, which can correctly sort out whether one int is bigger than another, regardless of combination of types, and while I don't like macros, this does the job:

template<typename T, typename U> bool SafeLhsMax(T t, U u) {SafeInt<T> st(t); return st > u;}

Followed up with:

#define TMax(a, b) (SafeLhsMax((a), (b)) ? a : b)

There's a bit more to it – we need some overloads in case someone passes a SafeInt into our macro, and it will object if we try to use a float, so we could make some specializations there, but there you have it – one of the more interesting bugs I've seen this month, and really points out that disabling warnings on a large scale is often a bad thing, which could lead to user astonishment, and even exploits. So compile your code with /W4 and /WX, and fix the warnings! SafeInt also has the happy side-effect of making the signed-unsigned comparison warnings go away, too. Now I just have to hope I don't hit any offsetting bug regressions where two mistakes really did add up to something that's OK, and now that we're down to one mistake, it's a bug.

PS – if you think you're safe because you run /analyze (the bits of prefast/OACR that are externally available), think again – they don't tend to re-implement compiler warnings, since if the compiler already does it, why should they?

Safebool

david_leblanc — Wed, 03 Oct 2007 21:33:00 GMT

My last post triggered a couple of responses and a URL I thought would be good to not get lost in the comments. Check out http://www.artima.com/cppsource/safebool.html.

As I was saying a couple of posts ago, the right tool is usually situational. In my case, things like SafeInt<int>(2) << 3, and int I = SafeInt<unsigned long>(SomeFunc()) not only make sense, but are explicitly supported, and because all of those things are already in place, it's OK to overload bool. They then go on to talk about operator !, which is also already supported, since you have to be able to get ![some integral type], and you want a SafeInt to behave as closely as possible to the integer type it's encapsulating.

The safebool trick is pretty neat, but you need to heed the warning about whether the comparison operators do or don't make sense for your class. They do make sense for a SafeInt (since I want to avoid thinking that -1 == 2^32-1), but probably wouldn't make sense for another class, though we do want to be able to compare pointers.

What I'm not so sure about is just why the safebool idiom won't compile in a SafeInt, but given the overall context, I think overloading bool is OK.

Something else I really like about the article is that he also talks about when NOT to use this trick – you'll run afoul of a lot fewer side effects by creating a isValid() method. Sure, it isn't cool, or elegant, and it won't mystify your friends, but there's a lot to be said for writing code that someone just out of school can understand and use properly.

At any rate, thanks for the comments and the pointer to the interesting article.

C++ operator overloading trivia

david_leblanc — Wed, 03 Oct 2007 02:14:00 GMT

Learned something interesting this week that I'll be working into SafeInt 3. It all started out because if you declare a SafeInt class instance, and then try to use it as an array index, the compiler can't figure out which of the several available integer casts to use for the index. According to the language, an array index should get a ptrdiff_t, which is a 32 or 64-bit signed int, so I'm not sure why the compiler doesn't ask for that, but I digress. I asked an internal alias with a bunch of C++ experts, and immediately got a bunch of questions about why I designed SafeInt to do things the way I did. About the time I was starting to regret asking the question, someone popped up with a question about why I overloaded the Boolean operators (&& and ||), and thought I could replace it with a casting operator to some weird function thing. The initial claim was that this certainly wouldn't cause any side-effects or problems, and given that SafeInt is highly complex, is used thousands of times across Office alone, and it's found more than one compiler bug (not to mention not compiling at all on compilers without really good, current C++ standard template support), I was skeptical to say the least. I was right to be skeptical, as his original suggestion wouldn't even compile at all, but it did lead us to something that's better.

Here's the problem:

#include <stdio.h>

int Bar( int c )

{

return printf("c = %d\n", c);

}

class PtrThing

{

public:

PtrThing() : m_pThing(0){}

bool operator &&( bool b ) const

{

return m_pThing && b;

}

void* m_pThing;

};

int main(int argc, char* argv[])

{

PtrThing p;

if( p && Bar(argc) )

printf("true\n");

return 0;

}

Without the overloaded && operator, we can't write a proper if statement using a PtrThing object. We could certainly write another overloaded operator, or a Ptr() method to extract the underlying pointer, and then pass that to an if statement. Given the design goal for SafeInt of being able to replace 'int' with SafeInt<int> and have it not only compile but do the right thing, it seemed reasonable to have an overloaded && and || operator.

However, there's a problem – we're used to things short circuiting. If the first part of a logical AND evaluates to false, we don't evaluate the second, and the converse is true of a logical OR. When you overload the && and || operators, the inputs have to be evaluated, which isn't the way things normally work, leading to programmer astonishment and bugs. This is why it is generally bad to overload these operators, but for SafeInt, it seemed better than not having a logical AND compile at all.

As it turns out, there's a better alternative – overload the cast to bool. Try taking out the operator && above, and add this:

operator bool() const { return !!m_pThing; }

It now not only compiles, but it short-circuits properly. There is a drawback – if I had some situation where a PtrThing could be unexpectedly cast to bool and that wasn't what I wanted, then we could have anything from an ambiguous compile, resulting in errors to an actual runtime bug.

I know this is fairly obscure, and only something that a C++ geek would care about, but I thought it was interesting and could have implications for certain types of classes, especially pointer containers.

Even More Cool Integer Tricks

david_leblanc — Mon, 02 Apr 2007 06:57:00 GMT

OK, so this is just utterly geeky, and would really only come in handy if you're writing something like SafeInt –

How to tell if a numeric template type is a bool at compile time:

isBool = ((T)1 == (T)2)

if type T is a bool, then this is true, else it's an int.

How to tell if a type is an int or a float:

isFloat = ((T)1.1 > (T)1)

This one's a little more obvious:

isSigned = ((T)-1 < 0)

SafeInt 3 is coming soon – got it all working with the test rig last week, fixed a couple of bugs.

Speaking of testing, this is how to do things – don't even think about testing this thing with code review – tried that, and even many educated eyes didn't find all the problems. Speaking of many eyes, SafeInt 1.0.7 and 2.0 have been public for a long time, and there were a couple of minor issues, but the many eyes haven't found anything. Good thing I have a really good tester. Anyway, the test harness is cool – I just plug in the new code to the test harness, it doesn't fuzz, but it does try every corner case you could ever find, and then I just fix bugs until the test harness quits complaining. Keeps me from introducing regressions, too. Even if you don't have team programming, writing a test harness to validate your code and throw some evil inputs at it while you're at it will seriously improve quality. It's why a lot of the time, I'll get something working outside of the Office build environment, then once I'm fairly sure it works, THEN I port it over.

More Fun with Integers

david_leblanc — Wed, 21 Mar 2007 18:45:00 GMT

Just a quick note this morning to share something I found while finishing up SafeInt 3.0. This is something more helpful with 64-bit porting than with general security, though it does have some security side effects. Warning - heavy C++ programming geek content ahead -

If you declare a function like so:

bool Foo( unsigned long ul )

And then pass in a size_t, you'll be truncating when you start compiling the code as 64-bit. The compiler will warn you about this, but if you had done:

unsigned long cbSomething;
bool Baz( unsigned short s );

And then passed cbSomething to Baz, it gets truncated, and the compiler won't be any help about it. Similarly, if we passed a signed short to Foo, it gets sign extended, then cast to unsigned, potentially changing the value. The compiler won't warn about that, either. Given that integer mayhem figures strongly in both security issues and is job #1 in terms of doing a port to 64-bit, this standard behavior is less than ideal.

One of the things people asked for in SafeInt was a way to make a boxed version that wouldn't blow up unless you unboxed an invalid object. We have a version of this working internally, and I decided to make it part of SafeInt 3.0. This meant I needed to rewrite a whole lot of the internal functions to work like this:

// Boxed version
template< typename T, typename U>
bool Op( const T& t, const U& u, T& ret ) throw()

// regular SafeInt version
template< typename T, typename U>
void Op( const T& t, const U& u, T& ret )

You might wonder why I'm passing references instead of passing the integers by value. I did this largely to make 64-bit types perform a little better on 32-bit platforms, though if you read the standard carefully, it says those get passed as pointers under the covers anway. At any rate, it is slightly more desciptive C++, it seems to inline just a little bit more nicely, and I definitely had to make the return value a reference, so why not be consistent?

As I found out once I started doing the work of getting the test harness to compile, if you pass something by reference that isn't the right _size_, the compiler will not only complain, it will throw ERRORS, and you get to think about what you're doing, and whether you want to really be truncating or extending. It did complain about some things that were benign, and I had to create a couple of internal temporary variables when I needed to modify the input, but it also found some real bugs.

Any time I can get the compiler to tell me about bugs, as opposed to having to step through the code trying to figure out why something went sideways, or worse, stepping through the code trying to figure out why the instruction at 0x41414141 isn't executing because there's an exploit, then I'll take the compiler every time.