Eugene Siu's Thoughts on Security

(In)Security of MultiByteToWideChar and WideCharToMultiByte (Part 2)

esiu — Sat, 15 Nov 2008 08:59:34 GMT

Part 1 of this installment discussed the unsafe nature of MultiByteToWideChar and WideCharToMultiByte. They do not guarantee terminating strings properly. In this installment, I want to focus on the count parameters. There are three count parameters that warrant your attention in order to use these two functions properly.

Since these two functions deal with conversion, the size of MultiByte and WideChar buffers are expected. In addition, upon successful return, the number of converted characters is also returned. These three counts must be carefully provided in order to ensure security.

Rules of thumb
1. MultiByte parameters always expect byte counts, the number of bytes in the corresponding buffer
2. WideChar parameters always expect character counts, the number of wide characters in the corresponding buffer
3. Return value follows rule #1 or #2, depending the type of output

Let’s study a bad example to illustrate some common errors.

wchar_t wstrTest1[] = L"123456789";
char strTest1[9];

int result = WideCharToMultiByte(CP_UTF8, 0, (LPCWSTR) wstrTest1, sizeof(wstrTest1), strTest1, sizeof(strTest1), NULL, NULL);

Using sizeof() seems reasonable for both parameters, as WideCharToMultiByte expects size of the corresponding buffers. sizeof() returns the byte count of a buffer. Following the rules of thumb, let’s find the bug.

The bug lies in the count of wstrTest1. It expects the number of wchar_t characters, rather than the byte count. The proper value should be character count, not byte count, therefore, sizeof(wstrTest1)/sizeof(wchar_t).

The correct call should be as follows:

int result = WideCharToMultiByte(CP_UTF8, 0, (LPCWSTR) wstrTest1,
sizeof(wstrTest1)/sizeof(wchar_t),
strTest1, sizeof(strTest1), NULL, NULL);

What is so bad about getting the count wrong? It is possible to cause buffer overflows. Considering MultiByteToWideChar, destination count is expected to be the count of wchar_t. If the actual size is used, destination size erroneously doubles. MultiByteToWideChar would gladly copy data up to double the size of the valid buffer. As a result, buffer overflow occurs.

(In)Security of MultiByteToWideChar and WideCharToMultiByte (Part 1)

esiu — Thu, 06 Nov 2008 21:22:38 GMT

There are a few well-known unsafe APIs in the standard C library, such as strcpy and memcpy. These routines are unsafe as buffer and destination buffer size are not taken into consideration. Buffer overflows may take place because destination buffer is not large enough to hold incoming data. Safe version of APIs checks that destination buffer has enough spaces to hold the source data. In addition, strcpy has another potential risk as it does not guarantee production of null-terminating strings.

A couple functions that are almost as dangerous are still lurking without corresponding safe versions. They are MultiByteToWideChar and WideCharToMultiByte, which are used for translating strings between Unicode and ANSI. http://msdn.microsoft.com/en-us/library/cc500362.aspx. Similar to strcpy, they are dangerous because they are not guaranteed to produce null-terminated strings. Unlike strcpy, they return 0 when destination buffer is not large enough. Even though they perform size checks, I will explain the security pitfalls of using MultiByteToWideChar and WideCharToMultiByte.

The first installment focuses on the possibility of producing non-null terminating strings. Let's start with explaining the return values of MultiByteToWideChar and WideCharToMultiByte. They return 0 when destination buffer is not large enough, or fails with other reasons. They return the number of characters saved in the destination buffer if execution is successful. Non-null terminated strings can be produced by both successful and unsuccessful execution of these APIs. Let's illustrate them with a couple examples. WideCharToMultiByte is used in all examples, as the same principle applies to MultiByteToWideChar too.

wchar_t wstrTest1[] = L"123456789";
char strTest1[9];

// Case 1: Parse 2 characters without a null-terminating character
int result = WideCharToMultiByte(CP_UTF8, 0, (LPCWSTR) wstrTest1, 2, strTest1, sizeof(strTest1), NULL, NULL);

    // Result: 2 (Success) strTest1: 2 characters long without null-terminated
    printf("Result: %d\n", result);
    printf("Converted: %s\n", strTest1);

// Case 2: Parse all characters, which are larger than the destination buffer
result = WideCharToMultiByte(CP_UTF8, 0, (LPCWSTR) wstrTest1, 10, strTest1, sizeof(strTest1), NULL, NULL);

    // Result: 0 (Fail) strTest1: 9 characters long without null-terminated
    printf("Result: %d\n", result);
    printf("Converted: %s\n", strTest1);

From these two code examples, it illustrates that both successful and unsuccessful execution can result in non-null terminated strings. In the first code example, WideCharToMultiByte is asked to process two characters without a trailing null character. WideCharToMultiByte successfully converts the first two characters, but does not terminate the output string with a null character. This case is somewhat acceptable as "garbage in, garbage out" principle applies.

However, the second case is worse. Destination buffer is not large enough to hold translated data. WideCharToMultiByte fails correctly by returning 0, but destination buffer is nevertheless filled with data without a null trailing character. If destination buffer is used without checking return value, a non-null terminating string will propagate through subsequent use of the string.

Some developers are aware of this issue, and they have an elegant way of handling the return value. They add a null character to the destination buffer at the index indicated by the return value.

wchar_t wstrTest1[] = L"123456789";
char strTest1[9];

// Parse 2 characters without a null-terminating character
int result = WideCharToMultiByte(CP_UTF8, 0, (LPCWSTR) wstrTest1, 2, strTest1, sizeof(strTest1), NULL, NULL);

// Some developers add this check
strTest1[result] = NULL;

    // Result: 2 (Success) strTest1: 2 characters long and null-terminated
    printf("Result: %d\n", result);
    printf("Converted: %s\n", strTest1);

When WideCharToMultiByte fails, it returns 0, and strTest1[0] is set to NULL. It is a reasonable outcome because failed execution should return an empty string. On the other hand, When WideCharToMultiByte succeeds, it returns the number of characters written to the destination buffer. strTest1[result] means that a null character is appended to the return string.

The above code works very elegantly in most cases, except a boundary case. It is always the boundary case, isn’t it? It has a subtle non-exploitable buffer overflow bug. What happen if the return value is equal to number of characters allowed in the destination buffer?

When that occurs, a null character is written past the end of the destination buffer. It can potentially destabilize the system and cause denial of service. There are many ways to fix this issue. A fix is to add a conditional statement to handle where return value is equal to the number of characters in the buffer. Another fix is to increase the size of destination buffer by 1 to accommodate the extra null character. This example illustrates the first fix.

wchar_t wstrTest1[] = L"123456789";
char strTest1[10];

    // Parse all characters
    int result = WideCharToMultiByte(CP_UTF8, 0, (LPCWSTR) wstrTest1, 10, strTest1, sizeof(strTest1), NULL, NULL);
    if (result == sizeof(strTest1))
        strTest1[result-1] = NULL;
    else
        strTest1[result] = NULL;

    // Result: 10 (Success) strTest1: 9 characters long and null-terminated
    printf("Result: %d\n", result);
    printf("Converted: %s\n", strTest1);

Developer Checklist

1. Always check the return values of MultiByteToWideChar and WideCharToMultiByte. If not, code reviewers should file them as bugs
2. Adopt the suggested fixes if you don’t need any custom error handling

There are more common pitfalls related to using these two APIs. It will be in the next installment.

My favorite security blogs and podcasts

esiu — Fri, 24 Oct 2008 04:18:04 GMT

What are your favorite security blogs or podcasts? Here are mine. Please leave yours in the comment section.

Podcasts

Security Now (http://www.grc.com/securitynow.htm)

CNet Security Bites (http://securitybites.cnet.com)

Blogs

Schneier on Security
http://feeds.feedburner.com/schneier/fulltext

Security Vulnerability Research & Defense
http://blogs.technet.com/swi/rss.xml

The Microsoft Security Response Center (MSRC)
http://blogs.technet.com/msrc/rss.xml

Dark Reading
http://www.darkreading.com/

The Security Development Lifecycle
http://blogs.msdn.com/sdl/rss.xml

Microsoft Hackers blog
http://blogs.msdn.com/hackers/

Mark Curphey - SecurityBuddha.com
http://securitybuddha.com/feed/

“Out of Band” security patch MS08-067

esiu — Fri, 24 Oct 2008 03:52:18 GMT

Out of Band security patch MS08-067 is released today. Microsoft strives to keep our monthly patch Tuesday release cycle so that enterprise administrators can plan ahead for their testing and deployment. When out of band is released, it must be very urgent due to serious ramifications or presence of known exploits in the wild. You should install this patch as soon as possible.

MS08-067 fixes a vulnerability in the Server service that would allow malicious code to remotely execute code on successfully exploited computers.

This impacts all versions of Windows. Bug severity is critical for platforms other than Windows Vista and Windows Server 2008. For Vista and Windows Server 2008, its severity is important.

For more information, please refer to http://www.microsoft.com/technet/security/bulletin/ms08-oct.mspx.

What is unique about patch Tuesday of October 2008?

esiu — Thu, 16 Oct 2008 02:48:00 GMT

Technorati Tags: Security

Every second Tuesday, MSRC releases security patches for Microsoft products that have fixed vulnerabilities. The best is to have no patches for patch Tuesdays, and many administrators can take a break from installing patches across their server farms and enterprise desktops. It will be a long road ahead before Microsoft can get to zero security bugs for all products.

Before reaching the elusive zero-bug milestone, Microsoft continues to innovate on security bug release process to better inform the public of those vulnerabilities so that administrators can make informed decisions on their patching schedule.

From October 2008 patch Tuesday, Exploitability Index will be associated with each bulletin. Exploitability index is a number from 1 to 3 to indicating exploitability, with 1 being the most serious in terms of exploitability.

Exploitability Index Assessment	Short Definition
1	Consistent exploit code likely
2	Inconsistent exploit code likely
3	Functioning exploit code unlikely

Here is another data point to measure risk and decide on priority of patching for any given bulletins.

esiu — Wed, 26 Mar 2008 03:14:00 GMT

Troubleshooting Networking and IPSec Issues

esiu — Mon, 05 Nov 2007 11:30:00 GMT

I had a very strange networking issue last weekend. After connecting to corpnet via VPN and direct hookup, I was able to ping all remote servers, but was not able to do anything, such as web browsing and remote desktop. It was not the first time that I faced this issue, and helpdesk told me that IPSec settings may have messed up.

The following set of commands has helped me flush settings related to networking and IPSec, and restored my corpnet connection in both situations.

1. Launch a DOS command prompt
2. netsh int ip reset all
3. netsh winsock reset
4. ipconfig /flushdns
5. nbtstat -RR
6. gpupdate /force

Akin to "Ctrl-Atl-Del" of your networking settings, this series of commands should flush IP interface, DNS, Winsock, NetBIOS and group policy settings.

Good luck to troubleshooting your networking and IPSec issues.

ASP.NET ValidateRequest does not mitigate XSS completely

esiu — Sat, 20 Oct 2007 00:26:17 GMT

As a security guy, I can safely say that there is no magic bullet to mitigate any security problems completely, and cross-site scripting(XSS) bugs are not exceptions. Since ASP.NET 1.1, ValidateRequest can be configured in web.config to check and reject dangerous inputs, and HttpRequestValidationException is thrown before the input is even processed by your code. For example, <script> would be caught by ValidateRequest.

During my security reviews, I routinely find that many web applications turn on ValidateRequest (It is on by default), and do not follow XSS mitigation techniques, such as output encoding by HTMLEncode or ACE Anti-XSS library. They believe that ValidateRequest can fix all XSS problems.

However, there are a couple downsides of relying on ValidateRequest:
1. ValidateRequest may miss some crafty inputs. Please read MS07-040 for a recent MSRC fix on ValidateRequest.

2. ValidateRequest cannot be turned on in all cases, as characters that trigger XSS may also be needed in valid user scenarios. For example, AJAX transmits XML blobs between client and server, but ValidateRequest will throw HttpRequestValidationException as it contains "dangerous" characters, such as < and >. Exchange 2007 OWA cannot run with ValidateRequest turned on.

In conclusion, ValidateRequest should be turned on if it does not block valid user scenarios. However, even with ValidateRequest turned on, it MUST not be regarded as a sure-fire way to mitigate XSS. Please read http://msdn2.microsoft.com/en-us/library/ms998274.aspx for full XSS mitigation.

Read Office Files as ZIP

esiu — Fri, 19 Oct 2007 22:33:38 GMT

It is interesting to me that Office 2007 Metro formats can be broken down as a ZIP file. To see this in action, you can pick an Office 2007 Metro file, such as XLSX and DOCX, and rename its extension with ZIP. Then open the renamed file with WINZIP. You will see that Office 2007 content is laid out neatly in a ZIP. Check out this sample picture.

Let me illustrate a picture from http://www.devx.com/MicrosoftISV/Article/34934

Is Microsoft Office Isolated Conversion Environment(MOICE) mocha on ice?

esiu — Fri, 19 Oct 2007 21:49:38 GMT

MOICE may sound like mocha on ice, but it is really a strong dark espresso shot offered by Office TWC team to jolt up security. Microsoft Office Isolated Conversion Environment (MOICE) is a new security tool that helps protect Office users from malicious documents. Office team strives to enhance their security, and MOICE is another evidence that they are committed to security.

MOICE converts Office 2003 documents of supported types to Office 2007 XML formats (Metro) in an isolated environment. Granted that the same conversion engine is used in the Microsoft Office Compatibility Pack, how does running in an isolated environment enhance security?

Think of wearing pads and helmets while playing American football. It is the nature of American football that players will get hit hard. Wearing pads and helmets does not change the nature of American football, but it does lessen the chance of inflicting serious injuries on players

The same principle applies to MOICE. MOICE does not alter the fact that malicious documents are out there to exploit vulnerable machines. MOICE is like a pad and a helmet to reduce the chance of Office softwares being being exploited. Exploitation may still happen, but isolated environment provided by MOICE reduces possible damages inflicted by the malicious documents.

In addition, applying MOICE is as simple as putting on a pad and a helmet. MOICE can be installed as a recommended update via Microsoft Update, and execution of ASSOC replaces regular rendering with MOICE. For more information, please visit http://support.microsoft.com/kb/935865.

Last but not least, MOICE is not a replacement of properly patching your machines. Now, go patch your machines, enjoy a cup of MOICE and most importantly, don't click on suspicious Office documents via emails.

True test of a security geek

esiu — Thu, 11 Oct 2007 11:18:33 GMT

If you chuckle at this comic strip, congratulations! You are a security geek. If you don't chuckle, it is never too late to become one. Read my blog more, and you will become one.

Thanks TechJunkie for forwarding.

Given enough eyeballs all bugs are shallow: True or False?

esiu — Thu, 11 Oct 2007 10:18:10 GMT

"Given enough eyeballs all bugs are shallow." I do agree if more right-minded folks look at a piece of code, it would help identify both security and non-security bugs. This premise is built on the assumption that all reviewers have the best intentions in mind. However, do all people have the best intentions in mind? If all do, we will not need law enforcement officials.

Obviously there will be some malicious and devious "eyeballs" out there. Rather than identifying bugs, they plant bugs in open source softwares. This attack is named "Cross-Build Injection". Fortify just published an article with reported incidents related to OpenSSH, SendMail and IRSSI. Check out http://www.fortifysoftware.com/servlet/downloads/public/fortify_attacking_the_build.pdf.

System.URI.AbsolutePath Vs Phishing Attack

esiu — Thu, 11 Oct 2007 06:09:06 GMT

Phishing attack can be caused by users inadvertently clicking on malicious links in emails or web pages, which then forward requests to malicious websites. A common phishing technique is to fake emails sent by well-known banks or merchants,, which contain malicious hyperlinks. Successful phishing attacks allow attackers to steal online user identities, install malwares on the users' computers, etc.

A careful user named Clever John checks each link before clicking it. Some common phishing emails contain links with IP addresses, for example, http://123.123.123.123/notice.aspx. Obviously, famous banks or merchants are not likely to use IP addresses on their public announcement emails. Clever John decides not to click on emails that contain those suspicious links.

What happen if the link is http://www.famousbank.com/notice.aspx?redirect=http%3a%2f%2f123.123.123.123%2fnotice.aspx? This time Clever John checks the link as usual, and sees that it is originated from http://www.famousbank.com. In addition, the email graphically looks very convincing. Little does Clever John know that he just gets attacked by a phisher who exploits a phishing security bug on http://www.famousbank.com, and malwares are installed on his machine quietly, as a result.

Let's analyze the code behind the phising security hole. Here is sample code that is susceptible to phishing attack:

String redirectsite = request["redirect"];
response.Redirect(redirectsite);

Basically, the code redirects the server response to a location specified by redirect parameter in the request. This is a common scenario, for example, redirecting to home pages upon logoff in the form of http://www.famousbank.com/logoff.aspx?redirect=home.htm. Unfortunately, since redirect parameter is not verified, attackers exploit the hole to redirect requests to any sites of their choice.

In order to fix this bug, the redirect parameter MUST be validated.
* If the response should only be redirected to links of the same site, please check for "//". If "//" is present, it should be deemed as an invalid request.
* If the response can be redirected to external sites, more intelligent filtering must be in place. Define all the valid sites that can be redirected, and allow to be redirected to only those valid sites.

Someone asked if System.URI.AbsolutePath can be used to strip hostname and port number as a mitigation. Then, redirect to the location returned by System.URI.AbsolutePath.

Let's analyze what System.URI.AbsolutePath does.

// link0.AbsolutePath returns "/default.aspx"
Uri link0 = new Uri("http://www.evil.com/default.aspx");

// link1.AbsolutePath returns "//www.evil.com/default.aspx"
Uri link1 = new Uri("http://www.evil.com//www.evil.com/default.aspx");

You may think "//www.evil.com/default.aspx" is not a valid address because it does not start with "http:". However, web browsers, such as IE and Fire, would gladly prepend "http:" in front of "//www.evil.com/default.aspx", and redirect to http://www.evil.com/default.aspx. In other words, AbsolutePath alone is not effective against phishing security hole.

Phishing Attack wins in this heads-on "System.URI.AbsolutePath Vs Phishing Attack" battle .

Web Service Security Guidance

esiu — Thu, 11 Oct 2007 02:56:27 GMT

I have just published a Technet article. This is geared for administrators and developers as an introduction to web service security. It contains lots of references that allow you to deepend your knowledge of web service security.

Please visit http://www.microsoft.com/technet/community/columns/sectip/st1007.mspx.

Your feedback is welcome.

More eyeballs for .Net Framework code

esiu — Fri, 05 Oct 2007 05:49:00 GMT

Microsoft will open up source code of .Net Framework to the public. It allows outsiders to review what is under the hood, and enables easier debugging of development projects around .Net Framework. .Net Framework code has been reviewed heavily, and developers can pick up coding best practices by reviewing source code of .Net Framework.

The following libraries are available:
* Net Base Class Libraries (including System, System.IO, System.Collections, System.Configuration, System.Threading, System.Net, System.Security, System.Runtime, and System.Text).
* ASP.Net (System.Web).
* Windows Forms (System.Windows.Forms).
* ADO.NET (System.Data).
* XML (System.Xml).
* Windows Presentation Foundation (System.Windows)

Source code is made available under the Microsoft Reference License, which is intended for developers to have read-only access. However, it does not allow modification or distribution.

An additional feature: It supports JIT code download for debugging in the final release of Visual Studio 2008. If set up properly, you don't need to download all source code on your box. You can get it when you need it transparently.

You can read this article for more details.

Eugene Siu's Thoughts on Security

(In)Security of MultiByteToWideChar and WideCharToMultiByte (Part 2)

(In)Security of MultiByteToWideChar and WideCharToMultiByte (Part 1)

Developer Checklist

My favorite security blogs and podcasts

Podcasts

Blogs

“Out of Band” security patch MS08-067

What is unique about patch Tuesday of October 2008?

alert()

Troubleshooting Networking and IPSec Issues

ASP.NET ValidateRequest does not mitigate XSS completely

Read Office Files as ZIP

Is Microsoft Office Isolated Conversion Environment(MOICE) mocha on ice?

True test of a security geek

Given enough eyeballs all bugs are shallow: True or False?

System.URI.AbsolutePath Vs Phishing Attack

Web Service Security Guidance

More eyeballs for .Net Framework code