Larry Osterman's WebLog

Confessions of an Old Fogey
Blog - Title

Fuzzy interfaces

Fuzzy interfaces

  • Comments 21

Mschaef took my challenge up, and submitted the Michal Zalewski article to Slashdot the other day, and the editors at Slashdot decided to run it.

I found the reactions on Slashdot (and my blog) to be rather fascinating.

First off, I'm not the person who did the research.  That was Michal, and his work was (as usual) quite good.  I'm flattered that some people thought that I was something more than the messenger, but...

The first poster on Slashdot, Eponymous Cowboy, got it totally right, and I'm really happy about that.  His comment was (paraphrased):

Since it may not be obvious to all readers, be aware that when you can make a program crash by feeding it bad data, you can typically further manipulate the data you are sending it to take control of the program. That means a security hole. This is how buffer-overruns work. You can't always do it, but you can think of each way you can crash a program as a little "crack" in its exterior. If you can figure out a way to pry apart the crack, you've got yourself a hole.

He's totally right.  And the number of people who seem to believe that it's better to crash than to fail on invalid input distresses me immensely. 

Some of the Slashdot comments indicated that Michal's test was somehow "unfair" because it tested things like null characters in the middle of strings.  The thing is, the bad guys are going to try that too.  And if your browser can't deal with it, then...

If we lived in a world where the bad guys were forced to write valid code, then that attitude would probably be ok, but that's not the world in which we live.  The bad guys aren't going to use valid HTML to attack your web browser.  They're going to use invalid HTML.  So it's critical that your browser deal with both valid AND invalid HTML.

Your browser doesn't always have to render the invalid HTML, but it needs to deal with it in SOME reasonable manner.  If it drops the connection and throws up a "This is not strict HTML, I won't render it" dialog box upon seeing its first invalid sequence, that's ok (it may not be a useful customer experience, but that's a different issue).  But it can't crash.

If it crashes, then you've just let the bad guys own your customers machine.  At a minimum, they're going to be able to launch a DoS attack against your customers.  At the worst, they're going to own your customers machines.

One of the basic tests that MUST be performed on any exposed interface is what is known as "fuzzing".  Michael Howard and David LeBlanc wrote a about it in their "Writing Secure Code", but essentially fuzzing an interface means taking the interface definition and varying the input.

So if I wanted to fuzz HTML, I'd start by taking legal HTML, but start injecting invalid characters into the input - I'd try null characters, characters outside the ASCII (0-127) range, illegal UTF8 characters.  I'd then try arguments to ATOMs that were really long (longer than 64K, for example).  I'd try eliminating trailing tags, or nesting tags really, really deeply.  Etc.

The interesting thing about fuzzing an interface is that it can be automated, as Michal pointed out.

The other thing about fuzzing is that it should be a basic part of any security testers arsenal.  Your code can't be considered to be secure unless its external interfaces have been fuzzed.

 

 

 

  • Grammar pedant:

    "Your code can't be considered to be secure unless it's external interfaces have been fuzzed."

    That should be "its" :)
  • Nitpicker.
  • <i>If we lived in a world where the bad guys were forced to write valid code, then that attitude would probably be ok, but that's not the world in which we live. The bad guys aren't going to use valid HTML to attack your web browser. They're going to use invalid HTML. So it's critical that your browser deal with both valid AND invalid HTML.</i>

    No, they're far more likely to use javascript interpreter bugs, vbscript interpreter bugs, and local zone scripting exploits.
  • Richard, why would they go to all that effort if they can use basic HTML to compromise your machine?

    You're making the same mistake that the /. people made. The bad guy will go after your softest attack spot. So you can't have ANY soft spots.

    If your HTML renderer has security holes, it's just as bad as if your scripting engine has security holes (and according to Eric Lippert, there haven't been any scripting engine bugs in MS's scripting engine in quite some time).
  • 10/20/2004 3:07 PM Larry Osterman

    > The bad guy will go after your softest
    > attack spot.

    Now you're making almost the same mistake that you accused Richard of, though slightly varied.

    The bad guys and gals can go after all kinds of attack spots. Whether or not you know that one kind of attack spot is easier today or more popular today or whatever, you cannot ignore harder ones because anyone who wants to pursue them still can.

    It is commonly said that, to reduce the likelihood of your car getting stolen, you only have to make it harder to steal your car than to steal the car next door. When a thief wants to steal one car and is easily willing to give up with one and go on to try the next one, that is true. When a lot of thieves (whether a gang or just a lot of independents) wants to steal as many as possible, suddenly your minimum protection is meaningless.

    Back to the base note:

    > I'd try null characters, characters outside
    > the ASCII (0-127) range, illegal UTF8
    > characters.

    You'd try illegal characters, legal characters, and illegal characters. Sure, all kinds of illegal and legal and illegal received byte sequences really do need to be tested, but that's kind of a weird way to say so.
  • Norman wrote:
    > The bad guys and gals can go after all kinds
    > of attack spots. Whether or not you know
    > that one kind of attack spot is easier today
    > or more popular today or whatever, you
    > cannot ignore harder ones because anyone who
    > wants to pursue them still can.

    Splitting hairs, Norman. Fact is, exploitable parser bugs are just as bad if not worse than jscript bugs, malformed jpeg bugs, etc. Richard's approach of "hey, you don't need to fix those bugs because there are other attack vectors which will be used first" is a rather lackadasical one.

    > You'd try illegal characters, legal
    > characters, and illegal characters. Sure,
    > all kinds of illegal and legal and illegal
    > received byte sequences really do need to be
    > tested, but that's kind of a weird way to
    > say so.

    Null characters are not illegal, Norman. That's one of the rules of Unicode - and the HTTP protocol. And ASCII. NUL is a valid character in pretty much everything but the C and C++ languages. If you don't believe me, check out the ASCII spec.
  • >And the number of people who seem to believe that it's
    >better to crash than to fail on invalid input
    >distresses me immensely.

    Don't take it seriously - it's Slashdot, not "OSS Developers-Only". While there may be a few sensible posters left, hardly anyone there any more actuallydoes any OSS development or knows what they're talking about.

    Hell, I used "!=" in a comment recently and got a bunch of replies from people who clearly didn't know what it meant (and were offended by what that my comment implied if you thought it meant equals).
  • This is scary. And it highlights a fundamental flaw in the perception that's used to evaluate mainstream software.

    Many believe that Microsoft software has a lot of bugs. They are correct. But what they don't realize, I think, is how many bugs it doesn't have. The latter is just as much a measure of quality as a standing bug count.

    Where Mozilla--a browser which is gaining market share--is concerned, I must ask: where's the forest for the trees in this issue?

    It's not enough for the browser to simply be security-tested. There has to be a security _mindset_ throughout the development team, to increase that count of "bugs that didn't happen." More so, there needs to be accountability to security issues. Where these come from, is beyond me.

    How can Mozilla Firefox claim to be "safer" (on its home page at mozilla.org), with what evidently amounts to no actual safety verification? As I understand it, these "fuzz" tests are pretty basic, and they are taking place independently _after_ the browser has been distributed in the millions.

    Where a real company is concerned, this should be the basis for liability. Or at least the public perception of raw naivety. Especially with regards to what software quality actually is--it is certainly more than the absence of Microsoft's influence.
  • When was the /last/ time you heard about someone doing the fuzzy markup challenge on all the browsers?

    /me waits for answer
    /me gives up

    You haven't heard of this before and it hasn't been done to browsers before, by a white hat who would publish anyway. The concept of this kind of test for any program or API is so old, the first articles written about the idea (in general for all programs) are now yellow and crumbling.

    What would the outcome have been had the test been done 2 years ago? 5 years ago? or 1994? It is kind of a lucky accident for Microsoft that they didn't get this particular PR black eye. It's not because the bought-in Spyglass code was so good.

  • ATZ Man,
    You're totally right. Microsoft started doing this testing a couple of years ago, when we woke up and realized the kind of problem we had. Now every network facing interface must be fuzzed before test has signed off on it, because the dangers are so great.

    Hopefully the rest of the industry will soon too. Michal did a great service to the entire community with his post. It's a shame that so many people jumped down his throat for doing it.
  • I'm just curious, have many of the security patches over the last few years for XP, IE, and such, been found by automatic fuzzing? Or has this mostly only been used on newly changed codebases during validation?
  • 10/20/2004 8:58 PM Simon Cooke [exMSFT]

    > Null characters are not illegal, Norman.
    > That's one of the rules of Unicode

    Of course.

    > - and the HTTP protocol.

    OK, in that case I stand corrected. Where some recent security patches adjusted the display of some strings that had null characters in the middle of the strings, I thought that they were adjusting the display of _illegal_ URLs so that victims would be less likely to be deceived by some spamming hackers. Now I understand that they were adjusting the display of _legal_ URLs, though still so that victims would be less likely to be deceived by some spamming hackers. Thank you.

    > If you don't believe me, check out the ASCII
    > spec.

    I don't need to look at a foreign character specification, domestic character sets have it too. It's even a single byte in all of JIS, EUC, and SJIS. But I thought (and I think it looks like Mr. Osterman thought) that HTTP specs didn't allow them in the middle of strings.

    Of course, programs still need to act safely (accepting or rejecting) strings regardless of whether the strings' bytes are legal or illegal. I agree with Mr. Osterman on that.
  • In other news this month's security hole in IE happens to be URL spoofing vulnerability found by Netcraft. Mr. Osterman was unavailable for comments on IE's handling of bad HTML.

    More details at http://news.netcraft.com/archives/2004/10/29/new_url_spoofing_flaw_found_in_internet_explorer.html
Page 1 of 2 (21 items) 12