Null Is Not Empty

re: Null Is Not Empty

Mike — Thu, 14 May 2009 20:09:36 GMT

" I am occasionally asked why C# does not simply treat null references passed to “foreach” as empty collections, or treat null strings as empty strings. "

Except that C# does treat nulls as empty strings sometimes:

Console.WriteLine("foo" + null + "bar");

This prints foobar, it doesn't print an empty string nor does it throw an exception.

Excellent point, I had forgotten that one. Which is odd, since rewriting the code generator that does those semantics was my first task when I joined this team.

Those (in my opinion unfortuate) semantics are imposed upon us by String.Concat; the addition operator is just a syntactic sugar for a call to String.Concat. The designers of String.Concat chose to treat null concatenation as empty string concatenation. Which means that (string)null + (string)null gives you an empty string in C#, bizarrely enough. -- Eric

re: Null Is Not Empty

ghenne — Thu, 14 May 2009 21:20:51 GMT

This is interesting. Years ago, a product call eMbedded Visual Basic, which used VBScript as its engine, had a constant of vbNullPtr that was used for API calls. Any idea what this value would pass?

re: Null Is Not Empty

Daniel Earwicker — Fri, 15 May 2009 01:34:24 GMT

What's the advantage of your special empty sequence versus Linq's Enumerable.Empty<T>()?

Good point. There is no advantage. It makes more sense to just use the standard one. I've updated the text. -- Eric

re: Null Is Not Empty

Thomas Levesque — Fri, 15 May 2009 01:38:17 GMT

@ghenne : it would probably be equivalent to IntPtr.Zero...

re: Null Is Not Empty

Blake Coverett — Fri, 15 May 2009 04:06:33 GMT

Aww, a nostalgic post for me - I'm the same Blake from the comment thread on the original VBScript post.

Over five years later, and this has consistently been one of my favorite Microsoft blogs. Thanks for all the great articles, Eric.

You're welcome, thanks for reading! -- Eric

re: Null Is Not Empty

Matt Sherman — Fri, 15 May 2009 05:40:42 GMT

Great stuff Eric, I don't think people treat nulls as valuable information often enough. A few more thoughts: http://clipperhouse.com/blog/post/Nulls-and-knowledge.aspx

re: Null Is Not Empty

Charlie — Fri, 15 May 2009 06:31:30 GMT

Thanks for another interesting post - I heartily agree that these are important distinctions for programmers to make.

I also noticed Blake's comment, and went back through some of the discussion between you two from the original post. Time permitting, I would love to see a post or two about your ideas on interviewing, since I'm currently learning how to give interviews myself. How do you attempt to test problem solving ability? As I'm sure you've found, it seems a lot harder than just testing knowledge.

I've written two articles about interviewing. See the "interviewing" archive button on the sidebar. -- Eric

re: Null Is Not Empty

AC — Fri, 15 May 2009 06:36:35 GMT

I often see this manifested (or not manifested correctly) in data capture scenarios. It's all well and good to have strict validation, but sometimes your hapless user just doesn't know what the chassis serial number is, etc.

Overzealous developers who shun nulls in the database end up at some point creating sentinel values which for obvious reasons doesn't make anything easier in the long run. Not only do you have a non-standard syntax, but you better be sure your sentinel is really never going to happen and doesn't ruin any computations in the process,

Null is that special value that is outside the set of all permissible values, and I think sometimes people just think it's only the runtime scolding you from using an uninitialized reference.

FYI To all the people that hate checking for null, you can always use the Null Object Pattern if it makes writing your domain code easier.

re: Null Is Not Empty

Thomas — Fri, 15 May 2009 07:01:02 GMT

I'm pretty new to C#, but this has been nagging me for a bit

why isn't int nullable in C#?

Nullable ints are nullable in C#. Non-nullable ints are not nullable. This seems like a sensible approach, no? -- Eric

why does it default to 0?

Well, what value would you prefer a non-nullable int to default to? -- Eric

int thisIsAnInt = null;

throws an error on build

The syntax for nullable value types in C# is to put a question mark after the type. Try "int? x = null;" -- Eric

re: Null Is Not Empty

Charlie — Fri, 15 May 2009 07:22:20 GMT

(replying to myself) Actually it looks like you _have_ already posted some other stuff specifically about interviewing, which was very interesting. Thanks again for the great blog.

re: Null Is Not Empty

Anders Borum — Fri, 15 May 2009 08:08:46 GMT

Thomas, you can do:

int? thisIsAnInt = null; which is equivalent to

Nullable<int> thisIsAnInt = null;

re: Null Is Not Empty

tunefs — Fri, 15 May 2009 08:51:54 GMT

@Thomas

int is a ValueType and null does not apply to ValueTypes. Eric recently wrote an article on ValueTypes and referrenced types. That article might be worth a little of your type. (Not that it actually answers your question but it's deducing from your questing I believe it holds valueable information for you).

re: Null Is Not Empty

Mark Rendle — Fri, 15 May 2009 09:03:41 GMT

I love the null coalescing operator. It's great for lazy initialisation:

private List<Order> orders;

public List<Order> Orders

{

get { return orders ?? (orders = LoadOrders()); }

}

I've got a post on my blog about making that thread-safe:

http://blog.markrendle.net/post/Lazy-initialization-thread-safety-and-my-favourite-operator.aspx

re: Null Is Not Empty

Andrey Shchekin — Fri, 15 May 2009 09:45:17 GMT

Why the coalescing operator does not support shortened form, '??=' ?

(and why there is not &&= and ||= as well?)

The answer to every "why is feature X not implemented?" question is the same. No one designed, implemented, tested, documented or shipped that feature. You're the first person to ever ask me about this particular one in the last five years, so apparently no mob of angry programmers is banging down the door to building 41 demanding that we implement them. :-) Design, implementation, testing and documentation is expensive; we try to only implement features people actually want. -- Eric

re: Null Is Not Empty

Stephan Leclercq — Fri, 15 May 2009 10:03:44 GMT

I do not agree that a string is a reference type. Strings are values just as integers are. It just happens that the .NET architecture implements them as referenced objects. That's why you had to add the "IsNullOrEmpty" kludge. A proper string can never be null, in the same was as an integer can never be null. Unless I explicitly want it, in which case I would declare it as "string? s", in the same way as I declare a nullable integer.

Ordinary collections are not values (as they are mutable objects) but immutable collections definitely are values and should be non-nullable by default.

I agree that immutable types are logically values, and it would have been nice to represent that in the type system. I also agree that it would have been nice to build in nullability/non-nullability from day one, instead of starting with non-nullable value types and nullable reference types, then adding nullable value types, and then never adding the fourth. The next time you design a brand-new type system, keep that in mind.

But as a practical matter, I'm afraid strings are reference types, and that there are good reasons for that. The pleasant fact that value types are of known size, and need not be garbage collected makes it difficult to make strings value types. Also, the fact that strings can be cheaply copied by reference instead of copying all their bits, as we do with value types, is a big perf win. Would you rather abandon these benefits in exchange for making strings value types? What's the compelling benefit of making strings into value types that pays for the massive loss of performance that would entail? -- Eric

re: Null Is Not Empty

silky — Fri, 15 May 2009 12:46:31 GMT

You shouldn't really talk about database null as compared to C# null. Database nulls (in MS SQL at least) are very annoying and they don't mean "data is missing" they more accurately mean "i have no idea what this is"

I don't understand the distinction you're drawing between "missing" and "unknown". -- Eric

As you probably know, in SQL "where a = null" and "where a <> null" return exactly the same set of rows. You need to use "is not null". God help us all if C# used _that_ approach.

There was considerable debate over that when nullable value types were added to C# and VB. C# chose the approach you approve of -- which makes the semantics of the equality operators inconsistent and broken, but easier to read in the common case. VB chose the approach you disapprove of -- to lift equality to nullable and be consistent about comparisons of null values. Personally I prefer VB's approach; it is less intuitive but more accurate and consistent. (Which is funny, because normally C# is the less inituitive but more precise language and VB is the more intuitive but less precise language.) -- Eric

re: Null Is Not Empty

anthony.tarlano — Fri, 15 May 2009 13:02:36 GMT

Eric I hope you can answer my question below.. tia,

@Mark in your post about thread safety your final solution is equivalent to :

"""

1: private volatile OrderCollection orders;

3: public OrderCollection Orders

4: {

5: get

6: {

7: return this.orders ?? Interlocked.CompareExchange(ref this.orders, new OrderCollection(), null);

8: }

9: }

"""

My question is whether that Interlocked.CompareExchange is really even necessary?

Since you are using the volatile modifier on the orders field, the compiler will see that the backing field orders is a volatile field.

I am not sure, but I think that will make the whole statement expression extending all the way to the end point of the statement, i.e. to the end of the return, have locked semantics..

As I said, I am not clear on this, but would not these two statements be equivalent with regard to thread safety...

stmt 1: return this.orders ?? Interlocked.CompareExchange(ref this.orders, new OrderCollection(), null);

stmt 2: return this.orders ?? new OrderCollection();

re: Null Is Not Empty

anthony.tarlano — Fri, 15 May 2009 13:09:38 GMT

oops, final statements should have been

stmt 1: return this.orders ?? ( this.orders = Interlocked.CompareExchange(ref this.orders, new OrderCollection(), null) );

stmt 2: return this.orders ?? ( this.orders = new OrderCollection() );

New and Notable 321

Sam Gentile's Blog (if (DeveloperTask == Communication && OS == Windows) — Fri, 15 May 2009 17:08:13 GMT

C#/VS2010 Null is Not Empty VS2010: On Triangles and Performance - It sure looks like the *very* soon Beta 1 will exhibit some great work on Outlining and Performance Parallel Tasks - new Visual Studio 2010 debugger window ASP.NET Tip #61: Did you know...How

Interesting Finds: May 15, 2009

Jason Haley — Fri, 15 May 2009 17:29:22 GMT

Interesting Finds: May 15, 2009

re: Null Is Not Empty

Filini — Fri, 15 May 2009 17:49:50 GMT

This is a bit unrelated to the topic, but I started with this (to see your topic "in action")

String a = null;
var b = a + null;
Console.WriteLine(b.Length);

Then I tried using different objects, like:

Form f1 = new Form();
var f2 = f1 + null;
Console.WriteLine(f2.Length);

I was expecting compilation errors ("adding" null to a Form? "Length" of a Form?), but instead it compiles and runs just fine. The output is:

System.Windows.Forms.Form, Text:

So, it turns out that in "var f2 = f1 + null;" var becomes a string, and calls ToString() on f1 to concat (my guess). Is it so? And if yes, why? Why am I able to add a Form to a null, and get a String? I'm probably missing something in how "var" works...

Though I applaud your experimental approach, rather than guessing at the semantics you might consider reading the spec, which states:

The binary + operator performs string concatenation when one or both operands are of type string. If an operand of string concatenation is null, an empty string is substituted. Otherwise, any non-string argument is converted to its string representation by invoking the virtual ToString method inherited from type object. If ToString returns null, an empty string is substituted.

Now, this bit is not perfectly accurate. Clearly in your "form" case neither operand is of type string. This bit really should say "when one or both operands can be implicitly converted to string and operator overload resolution chooses one of the built-in string concatenation operators".

As I noted in Mike's comment above, I had momentarily forgotten about this unfortunate fact about string concatenation. This is not how I would have done things, but this choice was imposed upon the language by the implementation of String.Concat. It would be awfully weird to have a language where + did one thing and String.Concat did another. -- Eric

re: Null Is Not Empty

Seth — Fri, 15 May 2009 17:58:25 GMT

Is there a vb.net equivalent for ...

foreach(Customer customer in customers ?? Enumerable.Empty<Customer>())

re: Null Is Not Empty

paul — Fri, 15 May 2009 18:40:34 GMT

I have a question about the construct:

foreach(Customer customer in customers ?? Enumerable.Empty<Customer>())

I haven't used the ?? operator before but the above syntax looks a little klunky to me. Take this specific example:

int[] data = new int[] { 1, 2 };

foreach ( int i in data ?? Enumerable.Empty<int>() ) {}

in the above case if I just include System and not an System.Linq I get a compiler error. Why should I have to include Linq specific code for this? The foreach is "Linq independent" so it seems to me that maybe there should be a new keyword or contextual keyword to make this a little cleaner? We have foreach. We have default. Maybe DefaultEmptyCollection<int> which uses an array :)

The "Enumerable" class is in the System.Linq namespace. That's where all the rest of the LINQ sequence operators are, so it's a sensible place. -- Eric

Paul.

re: Null Is Not Empty

Steven — Sat, 16 May 2009 00:18:34 GMT

"The pleasant fact that value types are of known size, and need not be garbage collected makes it difficult to make strings value types. Also, the fact that strings can be cheaply copied by reference instead of copying all their bits, as we do with value types, is a big perf win."

Eric, I don't agree with you. It's pretty easy to make a string a value type (but the value must internally always contain a reference to a char array). The value would then always have the same size as the size of a reference (32 bits on x86) and copying is safe and just as fast as copying an integer or a reference. I believe the real reason not to implement it as value type is because this would lead to large amounts of boxing, especially in CLR 1.0 applications, where there was no generics.

Well, sure, I suppose. But I don't understand the point. I mean, we could cut out all the character array rigamarole and just say that struct MyString { public String theRealString } is a "value typed string". What does that buy us?

Taking a storage location which can contain a 32 bit managed reference to a string and reinterpreting it as storage of MyString doesn't change anything germane, it just makes it harder to take advantage of the underlying ref type. We can take _any_ reference type and explicitly wrap a value type around the reference; that just makes it slightly harder to compare things by reference.

It doesn't change the fundamental fact that the _data storage_ is ultimately implemented using reference semantics. Essentially what this example highlights is that references are themselves values. References are already treated as value types; the interesting thing about them is that they refer to something, not that they're copied around by value.

Making string, or any type, a "shallow" value type is trivial -- so trivial that it's not very interesting. Such a beast still has the fundamental property of reference types: that it refers to something else. Making it deeply a value type, the way, say, int is, so that it refers to nothing, that's a what I meant by it being a lot more difficult. -- Eric

Just for fun, here is a non-nullable string implementation (named 'vstring') as value type :-)

public struct vstring : IComparable<vstring>, IEnumerable<char>, IEnumerable, IEquatable<vstring>

{

private readonly string value;

public vstring(string value)

{

this.value = value;

}

public vstring(char[] value)

{

this.value = new string(value);

}

// Never returns null.

public override string ToString()

{

return value ?? string.Empty;

}

public override int GetHashCode()

{

return this.ToString().GetHashCode();

}

public int CompareTo(vstring other)

{

return this.ToString().CompareTo(other.value ?? string.Empty);

}

public IEnumerator<char> GetEnumerator()

{

return this.ToString().GetEnumerator();

}

IEnumerator IEnumerable.GetEnumerator()

{

return this.GetEnumerator();

}

public bool Equals(vstring other)

{

return this.ToString().Equals(other.value ?? string.Empty);

}

public vstring ToLower()

{

return new vstring(this.ToString().ToLower());

}

public vstring ToUpper()

{

return new vstring(this.ToString().ToUpper());

}

public static bool Equals(vstring a, vstring b)

{

return a.ToString() == b.ToString();

}

public static bool operator ==(vstring a, vstring b)

{

return Equals(a, b);

}

public static bool operator !=(vstring a, vstring b)

{

return !Equals(a, b);

}

public static vstring operator +(vstring a, vstring b)

{

return new vstring(a.value + b.value);

}

re: Null Is Not Empty

Stephan Leclercq — Sat, 16 May 2009 09:39:45 GMT

>> Would you rather abandon these benefits in exchange for making strings value types?

Eric,

Don't get me wrong. I'm not saying that they shoud be *implemented internally* as value types (ie being allocated on stack, copied bit by bit every time the wind blows, etc.) but they should *appear to the programmer* as value types (ie never be null, and instead initialized by default to the empty string, etc...) That's one of the too rare things Borland Delphi does right :-)

Basically that's almost just syntactic sugar : every time you see a string declaration, initialize it to String.Empty instead of null, and catch every assignation of null to a string (including return statements.)

re: Null Is Not Empty

Steven — Sat, 16 May 2009 16:07:43 GMT

>> Well, sure, I suppose. But I don't understand the point.

There is no point in doing that and it won't buy us anything, expect that programmers would see strings as value types, which is the point Stephan Leclercq tried to make. While I understand Stephan’s point, I’m against doing this. While strings then would really represent a logical value, this might give developers the wrong impression that strings would be copied completely by value, which could never be the case, because this -as you said- would be disastrous for performance.

So I didn’t disagree on string being a reference type, I only disagreed on the arguments you gave against string not being a value type.

ps. I saw my code example 'exploded' on your blog. It now takes a lot of space, sorry for that.

re: Null Is Not Empty

Pavel Minaev [MSFT] — Sun, 17 May 2009 22:38:11 GMT

>> But as a practical matter, I'm afraid strings are reference types, and that there are good reasons for that. The pleasant fact that value types are of known size, and need not be garbage collected makes it difficult to make strings value types. Also, the fact that strings can be cheaply copied by reference instead of copying all their bits, as we do with value types, is a big perf win.

This does not preclude from String being a value type. It just has to be a value type that encapsulates a single reference to an internal "StringData" reference type, with the latter working exactly as System.String works today. So user sees a value type, with no null value, and the implementation still gets all the benefits of a reference types.

>> While strings then would really represent a logical value, this might give developers the wrong impression that strings would be copied completely by value

It wouldn't matter in the slightest. Since strings are immutable, there are no observable effects between a copying implementation, and a sharing implementation (well, except for Object.ReferenceEquals, but why would you care about that one?). So it doesn't really matter what impression developers get - it will be consistent with behavior either way.

re: Null Is Not Empty

eff Five — Mon, 18 May 2009 18:25:11 GMT

"The designers of String.Concat chose to treat null concatenation as empty string concatenation.

Which means that (string)null + (string)null gives you an empty string in C#, bizarrely enough"

MS SQL has the same behavior if you SET CONCAT_NULL_YIELDS_NULL OFF which happened to have burned me last week because two different apps set it differently.

re: Null Is Not Empty

Michael — Mon, 18 May 2009 21:04:46 GMT

"Explain the difference between Null, Empty, and Nothing" has been one my favorite interview questions for years. I've always enjoied the null/empty/nothing stares after I ask that question.

Does the answer to question really tell you much about the candidate? If they've claimed to be a VB expert then that will certainly tell you whether they are or not I suppose. But I try to ask interview questions that allow the candidate to demonstrate skills, intelligence or passion rather than testing domain-specific knowledge. I assume that anyone who is smart, skilled and gets stuff done can learn the domain. -- Eric

re: Null Is Not Empty

I. J. Kennedy — Mon, 18 May 2009 23:55:02 GMT

"The designers of String.Concat chose to treat null concatenation as empty string concatenation."

Hopefully these designers have been re-assigned. :^)

Seriously, not a good design decision in my humble opinion. It only masks problems and adds to the newbie confusion between null and the empty string. I guess it's too late to turn back though.

Null – не то же самое, что Empty

Блог Эрика Липперта (перевод) — Mon, 25 May 2009 01:22:14 GMT

Когда я начал этот блог в 2003 году, один из первых постингов был про отличия между Null, Empty и Nothing

re: Null Is Not Empty

t-scotmc — Wed, 03 Jun 2009 03:47:17 GMT

Now that there's int?, I really want things like IEnumerable!, which would never be null, so I can stop getting forced to deal with "no information" in the places where it's really not appropriate.

Then have the compiler force people to do something like this before calling me:

IEnumerable<T>! seq = annoying ?? Enumerable.Empty<Customer>();

Microsoft Research makes a language called Spec# which has a non-nullable ref type system; you might want to check it out. Also, the next version of .NET will have a "contracts" system whereby you can annotate your code with contracts that describe nullability; unfortunately, we are not integrating it directly into the language at this time, but still, it gets you a lot of the way there. -- Eric