Fabulous Adventures In Coding : Security

Sorry about the CAPTCHA

Eric Lippert — Thu, 10 Sep 2009 16:28:00 GMT

A quick metablogging note. Those of you who comment on this blog (6700+ comments and counting, thank you all) have probably noticed that it now has a CAPTCHA, that little "please prove you're a human" test before the comment is posted.

I understand why. The MSDN and TechNet blog sites are high-value targets for unwanted commercial advertisers, for attackers who wish to attempt to influence search engines to drive traffic to their sites, and for vandals. The people who run security for this site have their hands full; we've experienced some pretty serious denials of service based on ham-fisted spammer attacks. Adding a CAPTCHA to regulate comments massively slows down the rate of successful comment spam.

That said, I'm not thrilled about this. I find CAPTCHA-style solutions distasteful for several reasons:

The benign commenter -- precisely the kind of person we want to encourage -- is forced to do additional work. This is a small but nonzero disincentive to writing comments.
Sometimes mistakes will be made; providing new ways that computers can tell us on a daily basis that we are failures seems irksome.
The assumption of innocence is changed to an assumption of guilt; the benign commenter must prove their innocence. Every time I have to fill out a CAPTCHA I feel a small but real insult; I'm a trustworthy person, so trust me already. As Joel Spolsky once pointed out, it's like walking into a train station and the first thing you see is the NO SKATEBOARDING NO PANHANDLING NO THIS NO THAT NO THE OTHER THING sign. Its unwelcoming. It makes you feel attacked and guilty and reminds you that there is evil in the world.
There are accessibility concerns. Not everyone who uses computers has decent vision but that doesn't make them evil robots. They deserve as much as a chance as everyone else to contribute and have to overcome plenty of obstacles already; let's not throw more in their way.
And so on

So, sorry about that, commenters. I don't like it any more than you do, but there's not much I can do about it; I don't run the blog servers. The only thing I can control is how purple the text is.

What's the Difference, Part Five: certificate signing vs strong naming

Eric Lippert — Thu, 03 Sep 2009 16:48:00 GMT

Both strong naming and digital signatures use public key cryptography to provide evidence about the origin of an assembly, so that you can apply security policy to determine what permissions are granted to the assembly.

They differ most importantly not in their mathematical details, but in what problems they are intended to solve.

The purpose of a strong name is solely to ensure that when you load an assembly by name, you are loading exactly the assembly you think you are loading. You say "I want to load Frobber, version 4, that came from FooCorp". The strong name gear ensures that you actually load precisely that DLL, and not another assembly called Frobber, version 4, that came from Dr. Evil Enterprises. You can then set security policy which says "if I have an assembly from FooCorp on my machine, fully trust it." These scenarios are the only by-design purposes of strong names.

In order to achieve this, all that is required is that you know the public key token associated with FooCorp's private key. How you come to know that public key token is entirely your business. There is no infrastructure in place designed to help you get that information safely. You're just expected to know what it is, somehow. If evil people can trick you into believing that their key token is in fact FooCorp's key token, then you have a problem. You are expected to come up with some reasonable way to determine what FooCorp's real key token is.

The purpose of a digital signature from a publisher certificate is to establish a verifiable chain of identity and trust. The chain of trust goes from a hunk of code of unknown or uncertain origin up to a "trusted root" -- an entity which you have configured your operating system to trust.

You download some code, and the code has a digital signature with a certificate from FooCorp. You examine the certificate and it says "this program comes from FooCorp. The accuracy of this certificate is vouched for by VeriSign." Since VeriSign is one of your trusted roots, you now have confidence that this code actually did come from FooCorp.

Notice how much more complex the problem solved by digital signatures is. We're not trying to simply determine "is this hunk of code associated with this name or not?" Instead we're trying to determine where did this code come from, and who vouches for the existence of the company allegedly responsible, and should we trust that company?

The difference between strong names and digital signatures emphasizes what is hard about crypto-based security. The hard problem isn't the cryptography; that's just math. The hard problem is safely managing distribution of information about the keys and associating them with the correct entities. Strong names, because they attempt to solve a very small but important problem, do not have key management issues. Or, rather, they foist the key management problem off to you, the user. Digital signatures are all about trying to automate safe distribution of key information via certificates, in order to solve much more complex problems of trust and identity.

Alas, Smith and Jones

Eric Lippert — Thu, 04 Jun 2009 17:04:00 GMT

We have a feature in C# which allows you to declare a "friend assembly". If assembly Smith says that assembly Jones is its friend, then code in Jones is allowed to see "internal" types of Smith as though they were public(*). It's a pretty handy feature for building a "family" of assemblies that share common implementation details.

The compiler enforces one particularly interesting rule about the naming of friendly assemblies. If assembly Smith is a strong-named assembly, and Smith says that assembly Jones is its friend, then Jones must also be strong-named. If, however, Smith is not strong-named, then Jones need not be strong-named either.

I'm occasionally asked "what's up with that?"

When you call a method in Smith and pass in some data, you are essentially trusting Smith to (1) make proper use of your data; the data might contain sensitive information that you don't want Smith to misuse, and (2) take some action on your behalf. In the non-computer world an entity which you share secrets with and then empower to take actions on your behalf is called an "attorney" (**). That got me thinking, which usually spells trouble.

You want to hire an attorney. You go to Smith & Jones, Esqs. You meet with Smith, the trustworthy-looking man on the left of this photo:

Scenario (1):

You check Smith's ID. It really is Mel Smith. You are contemplating giving Smith a secret, and empowering Smith to act on your behalf with the knowledge of that secret. Smith says "I share my secrets with my partner Griff Rhys Jones, whose identity you may now verify, here is Jones and here is Jones' ID. Jones will also be acting on your behalf. Jones has full access to all secrets which are internal to this organization."

You decide that you trust both Smith and Jones, so you share your secrets with Smith. A series of wacky sketch comedy hijinks then ensues.

(Smith and Jones are both strong-named assemblies. That is, they have "ID" you can verify. Smith's internals are visible to Jones. Everything is fine.)

Scenario (2):

You check the ID. It is Mel Smith. Smith says "By the way, I share my internal secrets with everyone in the world who claims their name is Jones, I don't bother to check IDs, I just trust 'em, and you should too! Everyone named Jones is trustworthy! Oh, and that includes my partner over here, Jones. Good old Jones! No, you can't check Jonesie's ID. Doesn't have any ID, that Jones."

This is ludicrous. Obviously this breaks the chain of trust that you showed you were looking for when you checked Smith's ID in the first place.

(The compiler keeps you from getting into this terrible situation by preventing Smith from doing that; Smith, a strong-named assembly, can only state that it is sharing his internal details with another strong-named assembly. Smith cannot share its internals with any assembly named Jones, a weakly-named assembly in this scenario. We restrict Smith from exposing internals to weakly-named assemblies so that Smith does not accidentally create this security hole or accidentally mislead you into believing that Smith's internals are in any way hidden from partially-trusted code.)

Scenario (3):

You don't bother to check Smith's ID. In fact, you give your secrets and power of attorney to anyone named Smith, regardless of who they actually are or where they came from. The first Smith you pick off the street says "By the way, I'll give my internal secrets to anyone named Jones".

Do you care? Why would you? You're already giving secrets away to anyone named Smith! If a con artist wants your secrets, he can pretend to be Jones and take them from Smith, or he can pretend to be Smith and get them from you directly, but either way, the problem is that you are giving away secrets and power of attorney to strangers off the street.

(Smith and Jones are both weakly named assemblies, Smith exposes internals to Jones. This is perfectly legal, because if you don't care enough about who you're talking to to check IDs then what's the point of the security system preventing those random strangers from talking to each other?)

**************
(*) Fully trusted assemblies of course can always use reflection to see private and internal details of other assemblies; that is what "full trust" means. And in the new CLR security model, two assemblies that have the same grant set can see each other's private and internal details via reflection. But friend assemblies are the only mechanism that allow compile time support for peering at the internal details of another assembly.

(**) or "delegate", which obviously I want to avoid because that means something technical in C#.

Preventing third-party derivation, part two

Eric Lippert — Mon, 06 Oct 2008 17:50:00 GMT

If you find a class in a library that has all private/internal constructors, it is pretty clear that the author of the class is sending you the deliberate message that this class is not to be extended by you. In our previous example, it was not so clear. Obviously the author of the class in question did not think that extending the class was a by-design scenario (because if they did, their scenario tests would have immediately caught the problem) but it is not so clear that this restriction was deliberate, rather than accidental.

I say that writing code and writing mystery novels are two entirely different things; don't give the user of your code a twisty-turny puzzle to figure out with a surprise at the end. Make it obvious. If I wanted to make a class that was not third-party extensible by design, I'd not only make the constructors private, I'd slap one of these on the class:

[PermissionSet(SecurityAction.InheritanceDemand, Name = "FullTrust")

Meaning: “Only classes inside assemblies which the current user’s security policy fully trusts may extend my class.”

Or this:

[StrongNameIdentityPermissionAttribute(SecurityAction.InheritanceDemand,
PublicKey="00240000048000009400000006020000002400005253413100040000010001005" +
"38a4a19382e9429cf516dcf1399facdccca092a06442efaf9ecaca33457be26ee0073c6bde5" +
"1fe0873666a62459581669b510ae1e84bef6bcb1aff7957237279d8b7e0e25b71ad39df3684" +
"5b7db60382c8eb73f289823578d33c09e48d0d2f90ed4541e1438008142ef714bfe604c41a4" +
"957a4f6e6ab36b9715ec57625904c6")]

Meaning: “Only classes inside assemblies that were signed with the corresponding private key may extend my class.” (FYI, the binary blob there is an encoding of Microsoft's public key.)

You'll notice that I said that I would make the constructors inaccessible and put a security annotation in the metadata as well. Why both? Two reasons. First, defense in depth. Don't just make something impossible, make it impossible in multiple ways. That way, if you're wrong about one of them, you've still got all the others to back you up.

Second, because in this case, going with just the metadata solution is not enough.

This was surprising to me. I cut my teeth on the .NET security system back in its earliest incarnation; things have changed since then. The meaning I gave of the second attribute up there is subtly wrong. It should have said

“Only classes inside partially trusted assemblies that were signed with the corresponding private key, or any fully trusted assembly, may extend my class.”

In recent versions of .NET, "full trust means full trust". That is, fully-trusted code satisfies all demands, including demands for things like "was signed with this key", whether it actually was signed or not.

Isn't that a deadly flaw in the security system? No. Fully trusted code always had the ability to do that, because fully trusted code has the ability to control the evidence associated with a given assembly. If you can control the evidence,then you can forge an assembly that looks like it came from Microsoft, no problem. (And if you already have malicious full-trust code in your process then you have already lost -- it doesn't need to impersonate Microsoft-signed assemblies; it already has the power to do whatever the user can do.)

Credit where it's due: Special thanks goes out to .NET security guru Shawn Farkas for taking the time to explain to me the differences between various versions of the .NET security system.

Coming up: in anticipation of the PDC later this month, over the next few postings I'll be responding vaguely and overcautiously to recent newgroup postings about proposed language features for some fictitious, hypothetical, unnannounced language called "C# 4".

Preventing third-party derivation, part one

Eric Lippert — Fri, 26 Sep 2008 18:17:00 GMT

In this episode of FAIC, a conversation I had with a C# user recently. Next time, some further thoughts on how to use the CLR security system in this sort of scenario.

Him: I have this abstract class defined in one assembly:

// Assembly FooBar.DLL
public abstract class Foo
{
internal abstract void DoSomething();
// ...
}

I want to create a concrete class derived directly from Foo in another assembly, Blah.DLL.

Me: You are going to have to learn to live with disappointment.

Him: But I should be able to because I have public access to the class Foo!

Me: That statement is false. Having public access to a class absolutely does not mean that it is always legal to create a subclass in another assembly. There are any number of reasons why it might not be legal to create a subclass in another assembly; you happen to have run across just one of them. (There are plenty more -- for example, you could make a public class and mark all the constructors as internal. It's not possible to subclass that from another assembly either, since the derived class constructor does not have an accessible base class constructor that it can call.)

Him: You're right. In this case, I can’t provide an overriding implementation for the internal abstract method DoSomething() because it is not accessible. Therefore the compiler will give an error when I try to subclass Foo. How do I get around this?

Me: You don’t get around it. The type safety system is working as designed; don’t try to defeat it.

If you own the FooBar.DLL assembly then of course there are several ways you can do what you want. Two that immediately come to mind are (1) mark Blah.DLL as a friend assembly of FooBar.DLL using the InternalsVisibleTo attribute (2) change the accessibility of DoSomething to public, protected or protected internal.

Him: Why is it even allowed to have a nonextensible class like this in the first place?

Me: It is sometimes a good thing to make a class that cannot be extended arbitrarily. I have myself occasionally used a similar technique to ensure that no third party can subclass one of my public base classes, though, as we’ll see later, there are other, perhaps better ways.

Look at it this way: if the author of the class wanted you to be able to subclass it, they probably would have made it possible. Clearly they do not want you to subclass this class.

Him: My previous question has been thoroughly begged. Why would someone want to prevent a third party from subclassing?

Me: I can think of a few reasons. Here are two:

1) Designing a class to be extended effectively by third parties is work, and work requires effort. If the class is not intended to be extended by third parties, but must be unsealed (for internal extension, for example) then the implementer is faced with a choice: spend the time/money/effort unnecessarily, provide an extensible but brittle class, or prevent extension by some other method.

This trick is a cheap, easy and straightforward way to prevent extension by arbitrary third parties without preventing extension by internal classes. As we'll see next time, there are other ways you can do this too.

2) A class may need to be extensible internally and used externally, but must not be extended by third parties for security reasons. Imagine you have a method Transfer which takes two instances of your class BankAccount. Suppose BankAccount is an abstract class with two derived classes, CaymanIslandsAccount and SwissAccount. Do you really want arbitrary third parties able to make new objects which fulfill the BankAccount contract but were not implemented by you?

Again, you end up with a tradeoff – either implement Transfer so that it does type checks on the BankAccount passed to it (possibly expensive both in initial implementation and maintenance), implement Transfer so that it accepts any old thing (dangerous!), or prevent anyone from making an unknown kind of BankAccount (cheap, safe).

In general, good security design is “make them do something impossible in order to break the system”. (Or even better “make them do seven impossible things”.) By making it impossible for third parties to fulfill the contract, you add additional security to the system. (By no means enough security, but a little bit more.)

Him: Thanks, I see what's going on here. Clearly I got into this situation because this trick of using access modifiers to prevent extension is insufficient to convey the author of FooBar.DLL's intentions.

Me: Next time we'll talk about more obvious ways to state the intention of "please don't subclass this thing".

Tasty Beverages

Eric Lippert — Wed, 20 Aug 2008 00:09:00 GMT

“Diet Dr. Pepper tastes more like regular Dr. Pepper.”

That was a previous advertising slogan for Diet Dr. Pepper, my personal favourite source of both caffeine and phenylalanine; I’m drinking it right now as I write this.

The present slogan is the brain-achingly oxymoronic “Diet Dr. Pepper: There’s Nothing Diet About It” – really? Seems like one ought to change the name then, if the name of one’s product is so misleading as to require its complete and utter disavowal in the slogan.

But that’s not what I want to talk about today. I actually want to talk about predicates.

The word “predicate” is one of those slippery words that has multiple technical meanings depending on the domain, all related but subtly different enough that one really ought to carefully call out how one is using the term.

In C# programming a predicate is a function which takes a bunch of arguments and returns a Boolean. customer=>customer.Age < 21, for example.
In mathematical logic, a predicate is a function (or relation) which indicates the membership of a set. For example, “all integers x such that x is both even and prime” is a predicate for the set containing only the number 2.
In English grammar, a predicate is that part of a statement which claims to represent a fact about the subject. For example, in “Thucydides fought in the Peloponnesian War”, “Thucydides” is the subject and the rest is the predicate. This predicate happens to be true for the subject “Thucydides”, and false for a different subject, say, “Julius Caesar”.

What all of these things have in common is that a predicate is something into which you can "substitute" a value to obtain a result of either truth or falsity.

What on earth does this have to do with Diet Dr. Pepper tasting more like regular Dr. Pepper?

Is “tastes more like regular Dr. Pepper” actually a predicate? If it is then when it is applied to a subject it must produce a statement which can be classified as true or false. Let’s leave the subjective nature of taste aside for a moment; that’s not the fundamental logical problem here.

Rather, consider this utterance: “Diet Dr. Pepper tastes more like” Is “tastes more like” a predicate? Of course not. That utterance doesn’t make any sense. Tastes more like… what? This utterance cannot be classified as true or false for any subject, so the latter part of it cannot actually be a predicate.

But the same thing goes for “tastes more like regular Dr. Pepper”! Tastes more like regular Dr. Pepper… than what?

In order to actually be a predicate it needs more objects. For example, “Diet Dr. Pepper tastes more like regular Dr. Pepper than a pint of Guinness tastes like a mango lassi” is a statement which actually has a truth value. Perhaps a subjective and arguable truth value, but at least this sentence has the form of a statement with a subject and a real predicate now. The original slogan’s “predicate” isn’t really a predicate at all; one might think of it as a pseudopredicate.

Advertisers love pseudopredicates. Once you realize that they exist you see them all the time. Advertisers love them because they make no testable claim which could be shown to be false in a court of law. Rather, they rely on either the irrational belief that “more” anything means “better”, or upon your brain’s ability to fill in the rest of the objects which they intend you to infer.

In this particular case, I imagine that the crafters of this slogan intended your brain to fill in “Diet Dr. Pepper tastes more like regular Dr. Pepper than our previous formulation of Diet Dr. Pepper tasted like regular Dr. Pepper” – that is, they want to make the claim that the product has improved without making the admission that the previous formulation was less than delicious.

Or, perhaps they want you to fill in “Diet Dr. Pepper tastes more like regular Dr. Pepper than Diet Coke tastes like regular Coke” – that is, they want to assert that their product is superior to a competing product. This assertion is in my personal opinion true, but the Coca Cola company could potentially take issue with if stated baldly as an objective claim. By relying upon a pseudopredicate to make a slogan which actually is so malformed as to have no truth value at all, the copywriters duck these thorny issues. There are lots of ways that clever advertisers leverage our tendancies to "fill in the blanks" in order to sell products.

That’s not actually what I want to talk about today either. I actually want to talk about writing secure code.

What on earth does Diet Dr. Pepper tasting more like regular Dr. Pepper have to do with writing secure code?

The other day I got a question about the characteristics of a particular bit of source code obfuscation technology, which we shall call X; what the technology actually consists of and what the precise question was are irrelevant to this discussion.

I answered the question with a question; I asked why it was that the questioner wanted to use technology X. The answer was “To protect the source code”. Leave aside for the moment the fact that I could probably have deduced from the original query that the questioner was interested in protecting some resource. There’s a deeper problem here. In the utterance “technology X protects the source code”, is “protects the source code” a predicate, or a pseudopredicate?

It’s a pseudopredicate. There is an object missing. To make this a predicate, it needs to be something like “protects the source code from casual inspection and editing by snoopy people” – as it happens, this predicate was true for technology X. What I was rather worried about was that the questioner actually had in his head the predicate “protects the database administrator password that I’ve stuck into my source code from discovery and misuse by a determined and intelligent attacker”. That predicate happens to be utterly false for technology X. Because he was not actually stating a predicate that could be true or false of X I was unable to answer the guy's question about X.

I never, ever lock my car doors anymore. Why? I drive a soft-top convertible. One day I woke up to discover that someone had sliced open the top and unlocked the car. The two bucks in quarters I keep in the car for parking meters was a trivial loss compared to the hundreds my insurance company paid to get the top replaced and the hours of my time wasted in dealing with the situation. The locks are not a mitigation to that vulnerability at all! Locking my car doors makes it more prone to be damaged, not less.

I do, however, lock my house, to protect it against random people wandering in. However, the locks are hardly any mitigation to the vulnerability of the house to determined attack from a wily, hostile burglar. It would be foolish of me to say that “the locks protect my house” without mentioning the threat.

What I’m rambling on about here is this: the fitness of a particular security technology to mitigate a vulnerability can only be evaluated in the context of a stated threat against a stated resource. That’s because every security technology is designed to mitigate specific vulnerabilities to particular threats. When you’re evaluating the benefit of a particular security system, make sure that the predicates you are using to talk about the system are actually predicates, not pseudopredicates; state the threats.

Protected Member Access, Part Four

Eric Lippert — Fri, 02 May 2008 19:13:50 GMT

In Part Two I asked a couple of follow-up questions, the first of which was:

Suppose you were a hostile third party and you wanted to mess up the parenting invariant. Clearly, if you are sufficiently trusted, you can always use private reflection or unsafe code to muck around with the state directly, so that's not a very interesting attack. Any other bright ideas come to mind for ways that this code is vulnerable to tampering?

Before I get into some ideas for attacks, I want to re-emphasize the bit in the middle there: attacks which presuppose that the attacker is already an administrator on your machine are not interesting attacks. A number of people came up with attacks involving the attacker being able to run debuggers or even replace the entire CLR with a custom-built runtime.

Since there is no point in protecting code from hostile code which is already stronger than the security system itself, it's not very interesting to consider such attacks. The interesting question is how to protect yourself against hostile code which is weaker than the code you are attempting to protect. That's what attackers want to do -- they want to leverage vulnerabilities in your strong code to turn their weak code into strong code. If they already have strong code, they don't need to lure your code into doing something, they can just do it directly.

Anyway, that said, there are a number of flaws in my proposed implementation from a security perspective:

Eamon Nerbonne pointed out the implementation is not in the slightest thread-safe; hostile code which wished to mess up the parenting invariant could run adds and removes on the same objects on multiple threads at once, and when the smoke clears, pretty much any weird parenting arrangement you care to name could be the case.
Matt pointed out that the implementation relies upon a correct implementation of hashing/equality. If a hostile or buggy object provides an implementation of hashing which is inconsistent with its implementation of equality, it is possible to put stuff into a hash set and never get it back out again. The parenting invariant in this implementation relies upon being able to use hash sets correctly.
Jon Skeet pointed out that the implementation has no protections against parenting relationships which are locally sensible but not globally sensible, like a box containing itself.

All of these vulnerabilities could of course be mitigated without too much difficulty; the trick is in remembering to look for the vulnerability in the first place!

Getting the parenting relationship consistently acyclic is an interesting algorithmic problem. There are numerous ways to do it. Two sketches for your consideration:

1) Every time you add an item to a container, search the parent list of the container for the item. Hopefully containment relationships are shallow and this is not too expensive.

2) Only allow adding an item to a container if the item and the container are different but are themselves in the same immediate container. If you have a box in a bag, and you want to put a ball in the box, first put it in the bag, and then in the box.

Next time, some thoughts on immutability and parenting relationships.

Why Can't I Access A Protected Member From A Derived Class, Part Two: Why Can I?

Eric Lippert — Sat, 29 Mar 2008 02:30:43 GMT

This is a follow-up to my 2005 post on the same subject which I believe sets a personal record for the longest time between parts of a series. (Of course, I didn't know it was a series when I started it.) Please read the previous article in this series, as this post assumes knowledge of part one.

.......

OK, now that you've read that, it's clear why you can only access a protected member from an instance of an object known to be of a type at least as derived as the current context. You can therefore deduce the answer to the question asked to me by a (very polite) reader this morning: Why did this code compile in C# 2.0 but give an error in C# 3.0?

public abstract class Item{
private Item _parent;
public Item Parent {
    get { return _parent; }
    protected set { _parent = value; }
}
}
public class Bag:Item{
private List<Item> list = new List<Item>();
public void Add(Item item)
{
    item.Parent = this; // Error in C# 3.0
    list.Add(item);
}
}
public class Torch : Item { }

It compiled in C# 2.0 because the compiler had a bug. We forgot to enforce the semantics of "protected" access for the property setter. Though the compiler did not generate an error, it did generate code which would not pass the CLR verification check! The code would run if it happened to be fully trusted, but it was not safe to do so. We fixed the bug in C# 3.0 and took the breaking change.

This raises a more interesting point though. Now that we correctly prevent you from setting the "parent" reference in this manner, how would you implement the desired pattern? The reader wanted the following conditions to be met:

There are two kinds of items -- "leaf" items, and "container" items. Container items may contain either kind of item, so you could have a box containing a bag, which in turn contains a torch.
An item may be in a container, and if it is, its parent reference refers to that container.
These classes must be extensible by arbitrary third parties.
"Unauthorized" code must not be able to muck around with the parenting invariants.

The reader was attempting to enforce these invariants by making the parent setter protected. But even in a world where it is legal to access a protected member from an arbitrary derived class, that does not ensure that the invariants are maintained! If you allow any derived class to muck with the parenting, then you are relying upon every third-party derived class to "play nicely" and maintain your invariant. If any of them are buggy or hostile, then who knows what can happen?

When I'm faced with this kind of problem, I try to go back to first principles. Considering each design constraint leads us to an implementation decision which implements that constraint:

There are two kinds of abstract items -- items and containers. Therefore, there should be two abstract base classes, not just one.
Containers are items. Therefore, the container class should derive from the item class.
The parenting invariant must be maintained across all containers. Therefore the invariant implementation must be inside the abstract container base class.
Every possible container requires "write" access to the parent state of every possible Item. We want that access to be as restricted as possible. Ideally, we want it to be private. The only by-design way to get access to a private is to be inside the class.

And now a full solution becomes very straightforward:

using System;
using System.Collections.Generic;

public abstract class Item {
public Item Parent { get; private set; }
public abstract class Container : Item {
    private HashSet<Item> items = new HashSet<Item>();
    public void Add(Item item) {
      if (item.Parent != null)
        throw new Exception("Item has inconsistent containment.");
      item.Parent = this;
      items.Add(item);
    }
    public void Remove(Item item) {
      if (!Contains(item))
        throw new Exception("Container does not contain that item.");
      items.Remove(item);
      item.Parent = null;
    }
    public bool Contains(Item item) {
      return items.Contains(item);
    }
    public IEnumerable<Item> Items {
      // Do not just return items. Then the caller could cast it
      // to HashSet<Item> and then make modifications to your
      // internal state! Return a read-only sequence:
      get {
        foreach(Item item in items) yield return item;
      }
    }
}
}

// These can be in third-party assemblies:

public class Bag : Item.Container { }
public class Box : Item.Container { }
public class Torch : Item { }
public class TreasureMap : Item { }

public class Program {
public static void Main() {
    var map = new TreasureMap();
    var box = new Box();
    box.Add(map);
    var bag = new Bag();
    bag.Add(box);
    foreach(Item item in bag.Items)
      Console.WriteLine(item);
}
}

Pretty slick eh?

A couple questions for you to ponder:

1) Suppose you were a hostile third party and you wanted to mess up the parenting invariant. Clearly, if you are sufficiently trusted, you can always use private reflection or unsafe code to muck around with the state directly, so that's not a very interesting attack. Any other bright ideas come to mind for ways that this code is vulnerable to tampering?

2) Suppose you wanted to make this hierarchy an immutable collection, where "Add" and "Remove" returned new collections rather than mutating the existing collection. How would you represent the parenting relationship?

Next time, another oddity involving "protected" semantics. Have a good weekend!

How do I mitigate a SQL injection vuln?

Eric Lippert — Wed, 01 Nov 2006 23:32:00 GMT

Joel points out today that SQL injection vulnerabilities are common and bad, bad, bad. He does a good job of describing the attack but doesn't really talk about how to mitigate it.

When I advise people on how to close security holes like this I always tell them that closing the original hole is probably not enough. You don't want to make the attackers do one "impossible" thing because you just might be mistaken about what is actually impossible. Make them do three or four impossible things, and odds are good that it really will be impossible to cause harm.

To start with, run all strings that come from users through a string checker looking for anything out of the ordinary. If you expect a string to contain only letters, numbers and spaces, then write a regular expression that verifies that and get in the habit of rejecting all input that doesn't conform. That should make it impossible for attackers to put special characters like quotation marks in there.

Assume that a clever attacker will subvert that. Assume that you've made a mistake and forgotten to put a check in somewhere. Look for every place in the code that uses that user-supplied string. Don't stop at SQL construction. Anything that gets passed to JScript's eval could be an injection attack. Anything that gets echoed back to the user could be a cross-site scripting attack. Anything that gets written to disk could be an attempt to write a script onto the server's disk to trick an admin into running it. Eliminate as many of these as you can.

But how do you eliminate them? A great way to mitigate the risk of a SQL injection attack is to use stored procedures. Stored procedures ensure that only the query that you want to run actually runs. But they have nice properties in addition to being more secure against injection. They can be updated in the database, so that when the database structure changes, you change the stored procedure rather than searching through your code for SQL statement construction. And stored procedures often run faster because the database can optimize itself for them.

How else can we make it impossible for an attacker to run Joel's proposed attack? The attack Joel describes deletes stuff from the database, which is pretty unsubtle. A more subtle attack would consist of an update query which ups the user's available credit, or changes the prices of products, or some such thing. The database code should make use of the principle of least privilege. If you only expect it to look up results from particular database tables, then create a database account that only has those permissions, and then use that account from the server code. Don't connect to the database as admin if you don't need admin privileges; that's just asking for someone to abuse those privileges.

Furthermore, don't keep secrets in source code. For instance, put the name of the database server, the name of the database account, and the password in a registry key or an ACL'd file. Assume that attackers will obtain your source code. Keep machine names, employee names, keep anything sensitive that an attacker could use out of the source code.

How did Joel find out that the server was vulnerable at all? When the server helpfully told him that there was malformed SQL, and furthermore, what part of it was. Not only is this potentially a cross-site scripting attack (because user-supplied data is echoed back) but it is like waving a big sign explaining exactly how to construct an attack! Detailed error messages that describe the internal state of the server should only be given to users who have been authenticated by the server and are known to be server developers. Ordinary users should get a generic "something bad happened" error that explains nothing.

For Joel's proposed attack to succeed, everything has to go wrong. The server has to fail to validate input, then use it in an insecure way, then connect to the database as an administrator. Regrettably, many server-side web apps leave themselves wide open to these sorts of attacks. Eliminate all of these problems, not just the string concatenation. Remember, think like an evil person, assume the worst, and make evildoers jump through as many hoops as you possibly can. Defense in depth!

Why are base class calls from anonymous delegates nonverifiable?

Eric Lippert — Mon, 14 Nov 2005 22:28:00 GMT

I'm still learning my way around the C# codebase – heck, I'm still learning my way around the Jscript codebase and I've been working on it for nine years, not nine weeks. Here's something I stumbled across while refactoring the anonymous method binding code last week that I thought might be interesting to you folks.

Consider this simple program:

using System;
public delegate void D ();
public class Alpha {
public virtual void Blah() {
    Console.WriteLine("Alpha.Blah");
}
}
public class Bravo : Alpha {
public override void Blah() {
    Console.WriteLine("Bravo.Blah");
    base.Blah();
  }
}

Pretty straightforward so far. Any virtual call to Blah on an instance of Bravo does a non-virtual call to the base class implementation. Now let's do a similar thing from an anonymous delegate in Bravo:

public void Charlie() {
    int x = 123;
    D d = delegate {
      this.Blah();
      base.Blah();
      Console.WriteLine(x);
    };
    d();
  }

When you compile this thing up you get a crazy-sounding warning:

warning CS1911: Access to member 'Alpha.Blah()' through a 'base' keyword from an anonymous method or iterator results in unverifiable code. Consider moving the access into a helper method on the containing type.

And indeed, if you compile this up and run it through PEVERIFY.EXE, sure enough you'll get an unverifiable code warning. Unverifiable code requires full trust and is generally to be avoided – what is going on here?

This is an artefact of the way that C# realizes anonymous delegates. The anonymous delegate above captures both this and x, and therefore we actually generate code that would look something like this if you decompiled it:

public class Bravo : Alpha {
public override void Blah() {
    Console.WriteLine("Bravo.Blah");
    base.Blah();
  }
private class __locals {
    public int __x;
    public Bravo __this;
    public void __method() {
      this.__this.blah();
      __nonvirtual__ ((Alpha)this.__this).Blah());
      Console.WriteLine(this.__x);
    }
  }
public void Charlie() {
    __locals locals = new __locals();
    locals.__x = 123;
    locals.__this = this;
    D d = new D(locals.__method);
    d();
}
}

Of course there is no such thing in C# as __nonvirtual__, so really there is no way to decompile this thing back into real C#. Which is exactly the problem! What's happening here is that we are doing a non-virtual call on a this reference which is not the "real" this reference of the call.

That's a pretty suspicious programming practice. People who build object hierarchies that manipulate sensitive information have a reasonable expectation that the only way to call a base class implementation of a virtual method is from the derived class itself, not from some third-party method. Therefore the CLR treats this as unverifiable code, and therefore we have to issue a warning. (I suppose the CLR team could have made an exception for nonvirtual references from classes scoped to within the class in question, or for that matter, classes from the same assembly, but they didn't.)

Now, I was refactoring the code that generates the captured variable class, and so of course I wanted to write a few additional unit tests to make sure that I wasn't breaking anything. When I ran into this warning the first unit test that I wrote was actually a somewhat simplified version of the above:

public void Charlie() {
    D d = delegate {
      this.Blah();
      base.Blah();
    };
    d();
  }

This issues the warning above, but when I ran the generated executable through PEVERIFY I was surprised to discover that it verified just fine.

As it turns out, the compiler is clever about this one. It sees that the only captured variable is the this reference, and therefore it can optimize away the locals class. For this case it simply generates the anonymous method as just another method on the Bravo class. Since this is a method of Bravo, not a special locals class, the nonvirtual call is on the real this reference, so it is verifiable.

We decided to issue the warning even in this case because we thought that it would make C# seem really weird and brittle to suddenly start issuing the warning when you add an additional outer local variable reference to the anonymous delegate. Even when this doesn't actually generate nonverifiable code, it's a good idea to get in the habit of creating a helper method on the real class that does the nonvirtual access.

Of course, all of the stuff above is implementation details. You cannot rely upon future versions of C# to continue to generate anonymous methods in this manner. We probably will, but who knows what new features will be added to the CLR that might make it possible to not generate a bunch of hidden classes behind the scenes to do this work? Please do not attempt to do anything silly like reflecting upon the class to discover the hidden nested classes and use them; you're just asking for future pain if you do.

Do not use string hashes for security purposes

Eric Lippert — Mon, 24 Oct 2005 17:00:00 GMT

A recent question I got about the .NET CLR's hashing algorithm for strings is apropos of our discussion from January on using salted hashes for security purposes. (If you missed it, you can read it here: Part One, Part Two, Part Three and Part Four). The question was basically "my database of password hashes doesn't seem to work with .NET v2.0, what's up with that?"

To make a long story short, the answer is UNDER NO CIRCUMSTANCES SHOULD YOU USE THE .NET STRING HASH ALGORITHM (that is, String.GetHashCode()) FOR SECURITY PURPOSES. That is not what it was designed for. If it hurts when you do that then stop doing it!

The slightly longer version goes like this. Suppose you want to store some secrets in a database, but you only need to be able to confirm that the user knows the secret. As I discussed in my series on salted hashes, a hash is a commonly used tool for this task because a cryptographic hash has some nice properties. Namely, it is a fixed number of bits (in the 100's of bits range), small changes to input produce huge changes in output, and it is very difficult to go from the hash back to the original secret. Another nice property that I didn't call out in my earlier article is that there are industry-standard hash algorithms where you can be reasonably guaranteed that any two implementations will produce the same results when given the same set of bits.

The .NET CLR string hash algorithm has none of these nice properties, and therefore is completely unsuitable for a cryptographic hash function. Specifically:

The string hash algorithm was designed to be blindingly fast rather than hard to run backwards, so it is likely that a mathematically astute attacker will be able to rapidly deduce facts about the input knowing only the hash.

Worse, being only 32 bits, using brute force to find a message that produces a given hash becomes doable in an afternoon with a PC rather than a trillion years.

Finally, the string hash algorithm is not an industry standard and is not guaranteed to produce the same behaviour between versions. And in fact it does not. The .NET 2.0 CLR uses a different algorithm for string hashing than the .NET 1.1 CLR. If you are saving .NET 1.1 CLR hash values in a database then you will not be able to match them when you upgrade to 2.0. The hash algorithm was specifically NOT designed to be forward/backward compatible and we called that out in the documentation because we knew that it probably would not be. Unlike last week, I have no problem whatsoever with making breaking changes when we call out in the documentation that you cannot rely on version-to-version identical output for a function.

So please don't do that; you're just asking for a world of hurt if you do. Use SHA1 or MD5 or some other algorithm designed for that purpose. (Yes, I know that weaknesses have been discovered in these algorithms, but they are still orders of magnitude better than a hash designed for hash table balancing!)

How To Obtain The Name Of The Client From The ASP Server

Eric Lippert — Mon, 09 May 2005 22:18:00 GMT

Here's a question about client side vs. server side scripting that I got recently:

I want to get the machine name of the client the request is being made from. With ASP I can get the IP address using this code: ipaddr = Request.ServerVariables("REMOTE_ADDR") But I don’t know how to get the name of the machine. Is there something I could do from the client side?

No, the web browser client cannot determine the name of the machine for two reasons.

First, if it could then the client could be instructed to send the name of the machine to an evil server. Evil hackers would love to have an internet web page that harvested intranet machine names that they could then attack. Knowing the name of a machine is particularly useful for social engineering attacks -- if someone phoned me up claiming to be from our IT department, I'd be a lot more inclined to believe them if they knew the names of all my machines.

Second, look at it from the other way. Suppose the client magically figures out its name and sends it to the server. Why should the server trust the client? What stops an evil client from sending a bogus name to the server? Even if the client could send the name, the server can't make any decisions based on that name, so it's kind of useless.

Clients and servers should not trust each other. In the absence of authentication evidence, clients must assume that all servers are run by evil hackers and servers must assume that all clients are run by evil hackers. Once you accept that fundamental design principle then it becomes much easier to reason about client-server interactions. Think like an evil person!

Another developer who saw this question suggested running this code on the server:

name = Request.ServerVariables("REMOTE_HOST")

That's a good start but not the whole story. By default this doesn't actually give you the remote host -- it just gives you the IP address again. If you want this to actually give you the name of the remote machine then there's some additional work you have to do. Since we have the IP address then we can do a reverse DNS lookup to see if there is a friendly name associated with that address. Now the server is trusting not an arbitrarty client but rather a specific reverse DNS server.

Read this Knowledge Base article on how to configure your server to automatically do Reverse DNS lookups when the code above is called.

http://support.microsoft.com/default.aspx?scid=kb;en-us;Q245574

Note that this will make your server performance worse, and of course is not guaranteed to work if the client machine is disguising its identity via a firewall, etc.

A Face Made For Email, Part Two

Eric Lippert — Fri, 01 Apr 2005 16:45:00 GMT

One year ago this week I was the Channel Nine guinea pig -- I'm still not sure why, but for some reason The Scobelizer and his cohort chose me to be the first guy interviewed for their project. (Probably because I'm mostly harmless.)

Channel Nine has succeeded tremendously, and I'm very pleased to have been a part of it at its birth. Recently Charles and Robert tracked me down and cornered me in a room with a really big XBOX screen to talk about Visual Studio Tools For Office and how we use .NET code access security.

If you've got 53 minutes to kill and you want to hear me ramble on about what I love to talk about most, check it out.

You Want Salt With That? Part Four: Challenge-Response

Eric Lippert — Mon, 07 Feb 2005 18:28:00 GMT

My friend Kristen asked me over the weekend when I was going to stop blogging about crypto math and say something funny again. Everyone's a critic!

Patience. my dear. Today, the final entry in my series on salt. Tomorrow, who knows?

***********************

So far we've got a system whereby the server does not actually know what your password is, it just has a salted hash of your password. But we're still sending the password over the wire in clear text, which seems risky.

System #4

What if the hash goes over the wire? The client sends the user name to the server. The server sends the password salt to the client. The client appends the password to the password salt, hashes the result, and sends the hash to the server. The server compares the hash from the client to the hash in the user list. Now the password never goes over the wire at all. Awesome!

Unfortunately, this system is worse. In previous systems the eavesdropper got the password; in this system the eavesdropper gets the salted hash. The eavesdropper can then write their own client which sends that username and salted hash to the server.

And the "steal the password list" attack just came back; now an attacker who gets the password list gets all the salted hashes, and can use them to impersonate the user. Sure, the attacker will still find it hard to deduce the original password from the salted hash, but he doesn't need to deduce the password anymore. (Unless of course the attacker is attempting to deduce your password on one system so that he can use it to attack another system that you use. This is why it's a bad idea to use the same password for two different systems.)

Essentially we've turned the salted hash into a "password equivalent". Can we fix this up?

System #5

How about this?

The client sends the username to the server. The server creates a second random salt which is NOT stored in the user list. This random salt is used only once -- we either make it so big that odds of generating it again are low, or keep a list of previously used random salts and pick a new one if we have a collision. We'll call the random salt "the challenge" for reasons which will become apparent in a minute.

The server sends the user's password salt and the challenge to the client. The client appends the password salt to the password and hashes the salted password. It converts the salted hash to a string, appends the string to the challenge, and hashes the resulting string to form the "response" hash. The response is sent across the wire.

The server then does the same thing – converts the stored salted password hash to a string, appends it to the challenge, and hashes the resulting string. If the response from the client is equal to the value the server just computed, then the client must have computed the same salted hash, and therefore knows the password.

Now what does the eavesdropper know? The eavesdropper knows the username, the password salt, the challenge and the response. The eavesdropper has enough information to launch an offline dictionary attack against that user. But since the random challenge is never going to be used again, the fact that the attacker knows a valid challenge/response pair is essentially irrelevant.

This system has the downside that an attacker who gets the password file has obtained password equivalents, so no dictionary attack is necessary. (Unless of course the attacker is trying to determine a user's password in order to try it against the user's account on a different system!)

Fortunately, these weaknesses can be mitigated somewhat by changing your password frequently, not using the same passwords for different accounts, never using common dictionary words as passwords, and making passwords long -- passphrases are better than passwords.

***********

This general challenge-response pattern is quite common in authentication systems that rely upon shared secrets, because at no point is the original secret password ever actually stored or transmitted! With such a system, a machine that does not know your secret can verify that you do know it -- almost seems magical, doesn’t it? Of course, now that you know how it works, it's not quite so magical -- the salted hash is essentially the shared secret, and it is stored and transmitted.

Clearly we could go on, adding more and more layers of encryption to this system to further mitigate these vulnerabilities, but I'm going to stop here. Readers interested in ways to solve problems such as mutual authentication (where the server authenticates the client AND the client verifies that it is talking to the right server) or other heavy-duty authentication tasks should read this charming dialogue on the design of the Kerberos authentication system: http://web.mit.edu/kerberos/www/dialogue.html

The foregoing is of course just a sketch with lots of details glossed over and greatly simplified for didactic purposes. Real professional-grade authentication systems do not work exactly like any of my sketches, though several are quite similar.

Unix boxes used to typically use a 12 bit salt and hash it with a 56 bit password using a DES-based hash, for instance. That's pretty weak! A 12 bit salt only makes construction of a dictionary take 4096 times longer -- one dictionary for each possible salt. In modern systems the more secure MD5 hash is often used now, which supports arbitrarily large salts. Unix boxes also used to keep the user list in the clear, so that anyone could read the salted hashes of the passwords; nowadays the encrypted passwords are kept in a file that can only be read by the operating system itself -- defense in depth!

NT by contrast uses an unsalted 128 bit MD5 hash to store the password equivalent, so it is susceptible to dictionary attacks should the password file be stolen. NTLMv2 is an authentication system based on some of the challenge-response ideas from System #5. It hashes the password, user name, domain, and current time in a fairly complex way; I'll spare you the details. And of course, many NT and unix boxes use Kerberos-based authentication these days.

You Want Salt With That? Part Three: Salt The Hash

Eric Lippert — Thu, 03 Feb 2005 14:53:00 GMT

Last time we were considering what happens if an attacker gets access to your server's password file. If the passwords themselves are stored in the file, then the attacker's work is done. If they're hashed and then stored, and the hash algorithm is strong, then there's not much to do other than to hash every string and look through the password file for that hash. If there's a match, then you've discovered the user's password.

You don't have to look through the vast space of strings in alphabetical order of course. An attacker will start with a dictionary of likely password strings. We want to find some way to make that attacker work harder. Setting a policy which disallows common dictionary words as passwords would be a good idea. Another technique is to spice up the hashes a bit with some salt.

System #3

For every user name we generate a random unique string of some fixed length. That string is called the “salt”. We now store the username, the salt and the hash of the string formed by concatenating the user’s password to the salt. If user Alpha's password is "bigsecret" and the salt is "Q3vd" then we'll hash "Q3vdbigsecret".

Since every user has their own unique random salt, two users who happen to have the same password get different salted hashes. And the dictionary attack is foiled; the attacker cannot compute the hashes of every word in a dictionary once and then check every hash in the table for matches anymore. Rather, the attacker is going to have to re-hash the entire dictionary anew for every salt. A determined attacker who has compromised the server will have to mount an entire new dictionary attack against every user’s salted hash, rather than being able to quickly scan the list for known hashes.

Salting essentially makes it less feasible to attack every user at once when the password file is compromised; the attacker must start a whole new attack for each user. Still, given enough time and weak passwords, an attacker can recover passwords.

In this system the client sends the username and password to the server, the server appends the password to the salt, hashes the result, and compares the result to the salted hash in the table.

This answers the original question posed by the JOS poster; the salt can be public because it is just a random string. Ideally, both the salt and the salted hash would be kept private so that an attacker would not be able to mount a dictionary attack against that salt. But there is no way to deduce any information whatsoever just from the salt.

And of course, it's better to not get into this situation in the first place -- don't allow your password list to be stolen! But it's a good idea for a security system to not rely on other security systems for its own security. We call this idea "defense in depth". You want to make the attacker have to do many impossible things to compromise your security, so that if just one of those impossible things turns out to be possible after all, you're not sunk.

But what about the fact that the password goes over the wire in the clear, where anyone can eavesdrop? That's now the weak point of this system. Can we do something about that? Tune in next time and we'll see what we can come up with.