References are not addresses

References are not addresses

Rate This
  • Comments 76

[NOTE: Based on some insightful comments I have updated this article to describe more clearly the relationships between references, pointers and addresses. Thanks to those who commented.]

I review a fair number of C# books; in all of them of course the author attempts to explain the difference between reference types and value types. Unfortunately, most of them do so by saying something like "a variable of reference type stores the address of the object". I always object to this. The last time this happened the author asked me for a more detailed explanation of why I always object, which I shall share with you now:

We have the abstract concept of "a reference". If I were to write about "Beethoven's Ninth Symphony", those two-dozen characters are not a 90-minute long symphonic masterwork with a large choral section. They're a reference to that thing, not the thing itself. And this reference itself contains references -- the word "Beethoven" is not a long-dead famously deaf Romantic Period composer, but it is a reference to one.

Similarly in programming languages we have the concept of "a reference" distinct from "the referent".

The inventor of the C programming language, oddly enough, chose to not have the concept of references at all. Rather, Ritchie chose to have "pointers" be first-class entities in the language. A pointer in C is like a reference in that it refers to some data by tracking its location, but there are more smarts in a pointer; you can perform arithmetic on a pointer as if it were a number, you can take the difference between two pointers that are both in the interior of the same array and get a sensible result, and so on.

Pointers are strictly "more powerful" than references; anything you can do with references you can do with pointers, but not vice versa. I imagine that's why there are no references in C -- it's a deliberately austere and powerful language.

The down side of pointers-instead-of-references is that pointers are hard for many novices to understand, and make it very very very easy to shoot yourself in the foot.

Pointers are typically implemented as addresses. An address is a number which is an offset into the "array of bytes" that is the entire virtual address space of the process (or, sometimes, an offset into some well-known portion of that address space -- I'm thinking of "near" vs. "far" pointers in win16 programming. But for the purposes of this article let's assume that an address is a byte offset into the whole address space.) Since addresses are just numbers you can easily perform pointer arithmetic with them.

Now consider C#, a language which has both references and pointers. There are some things you can only do with pointers, and we want to have a language that allows you to do those things (under carefully controlled conditions that call out that you are doing something that possibly breaks type safety, hence "unsafe".)  But we also do not want to force anyone to have to understand pointers in order to do programming with references.

We also want to avoid some of the optimization nightmares that languages with pointers have. Languages with heavy use of pointers have a hard time doing garbage collection, optimizations, and so on, because it is infeasible to guarantee that no one has an interior pointer to an object, and therefore the object must remain alive and immobile.

For all these reasons we do not describe references as addresses in the specification. The spec just says that a variable of reference type "stores a reference" to an object, and leaves it completely vague as to how that might be implemented. Similarly, a pointer variable stores "the address" of an object, which again, is left pretty vague. Nowhere do we say that references are the same as addresses.

So, in C# a reference is some vague thing that lets you reference an object. You cannot do anything with a reference except dereference it, and compare it with another reference for equality. And in C# a pointer is identified as an address.

By contrast with a reference, you can do much more with a pointer that contains an address. Addresses can be manipulated mathematically; you can subtract one from another, you can add integers to them, and so on. Their legal operations indicate that they are "fancy numbers" that index into the "array" that is the virtual address space of the process.

Now, behind the scenes, the CLR actually does implement managed object references as addresses to objects owned by the garbage collector, but that is an implementation detail. There's no reason why it has to do that other than efficiency and flexibility. C# references could be implemented by opaque handles that are meaningful only to the garbage collector, which, frankly, is how I prefer to think of them. That the "handle" happens to actually be an address at runtime is an implementation detail which I should neither know about nor rely upon. (Which is the whole point of encapsulation; the client doesn't have to know.)

I therefore have three reasons why authors should not explain that "references are addresses".

1) It's close to a lie. References cannot be treated as addresses by the user, and in fact, they do not necessarily contain an address in the implementation. (Though our implementation happens to do so.)

2) It's an explanation that explains nothing to novice programmers. Novice programmers probably do not know that an "address" is an offset into the array of bytes that is all process memory. To understand what an "address" is with any kind of depth, the novice programmer already has to understand pointer types and addresses -- basically, they have to understand the memory model of many implementations of C. This is one of those "it's clear only if it's already known" situations that are so common in books for beginners.

3) If these novices eventually learn about pointer types in C#, their confused understanding of references will probably make it harder, not easier, to understand how pointers work in C#. The novice could sensibly reason "If a reference is an address and a pointer is an address, then I should be able to cast any reference to a pointer in unsafe code, right?"  But you cannot.

If you think of a reference is actually being an opaque GC handle then it becomes clear that to find the address associated with the handle you have to somehow "fix" the object. You have to tell the GC "until further notice, the object with this handle must not be moved in memory, because someone might have an interior pointer to it". (There are various ways to do that which are beyond the scope of this screed.)

Basically what I'm getting at here is that an understanding of the meaning of "addresses" in any language requires a moderately deep understanding of the memory model of that language. If an author does not provide an explanation of the memory model of either C or C#, then explaining references in terms of addresses becomes an exercise in question begging. It raises more questions than it answers.

This is one of those situations where the author has the hard call of deciding whether an inaccurate oversimplification serves the larger pedagogic goal better than an accurate digression or a vague hand-wave.

In the counterfactual world where I am writing a beginner C# book, I would personally opt for the vague hand-wave.  If I said anything at all I would say something like "a reference is actually implemented as a small chunk of data which contains information used by the CLR to determine precisely which object is being referred to by the reference". That's both vague and accurate without implying more than is wise.

  • Eric, when are you going to write "Fun with Pointers: 101 ways to make your intern's head explode" ?

  • "a reference is actually implemented as a small chunk of data which contains information used by the CLR to determine precisely which object is being referred to by the reference"

    Yuck.  What useful information does this definition impart to the novice programmer?

    The fact is that OOP is an abstraction on top of many more abstractions, going all the way down to assembler, registers, the MMU, etc.  As with any abstraction, there are leaks, which *eventually* need to be understood to avoid certain problems or maximize efficiency.  But the value of the abstraction is in delaying the explanation of the gory details until after the fundamental concepts have been presented and understood.  Once you've reached that point, you at least have a context in which to talk about the leaks, and you can choose how deep you want to go, but it's certainly not material for chapter one or two unless the title of your book is something like "Advanced C# Concepts for the Experienced Programmer."

    For someone just starting to learn about OOP, I would stick with diagrams and pictures.  Draw arrows pointing from variables to stamped out object instances, and from object instances to other object instances, circle them in red, and say "That's a reference.  It lets you <i>refer</i> to object instances."  It might not be very technical or in-depth, but it effectively conveys the information that the reader needs to know at that particular point in time, without requiring them to understand 15 other concepts first.

  • I haven't these comments but I disagree with the original post.

    "References store the address of an object" means exactly what it says. It does not mean "references are pointers" or "references are addresses". You're munging the two texts.

    It would be an "implementation detail", documentation-wise, to say (correctly) that "references internally store the address of an object, although that detail is hidden from you, it only does this as an implementation detail to map the reference to the memory space that the garbage collector has allocated for it". But those are too many words; it is 100% truthful to shorten that down to simply say "references store the address of an object".

    If indeed the text said "references are addresses to an object", a description that comes closer to describing a C# pointer, I'd agree with your point.

    Also, I believe that even if a reference working with the garbage collector to retain the address to the memory space of an object detail is only an "implementation detail", I think it's a very important implementation detail to know. This sort of documentation usually comes up in the discussion of stacks vs. heaps, value types vs. reference types, and/or memory optimization in the CLR. It might not be all that interesting from a syntactical point of view (C# being a language), it is a very important thing to know from an architectural point of view (CLR being a runtime).

  • While it is a subtle distinction, I have always defined a reference and a pointer as follows:

    Pointer: A type of variable whose value represents the memory address of the instance of an object or type.

    Reference: A type of variable that acts as a synonym to represent an instance of an object or type.

    They sound the same as you read them, but I think the distinction for me is where they are significant. A reference to me is the developer telling the compiler that whenever he/she refers to X, it should interpret that is meaning Y. It's true that references are often implemented using memory address and often represent pointers, but it does not have to be. Perhaps though some clever use of Macros in languages like C, C++, you could implement something similar?

    I think it is hard to seperate the two concepts beacuse their implementations are almost always identical behind the scenes. But something that helped me understand that there was a difference is how they are treated with C++:

    A reference MUST be initialized to an object, while a pointer can be null. You can have a pointer exist that does not have a value (i.e. memory address). But you cannot have a reference exist that doesn't refer to another instance - because conceptually, a reference does not have a value.

    On the other hand, a pointer is and *always* will be a variable that contains a memory address of an object or type. You can rely on any implementation of a pointer always containing a memory address.

  • For beginner programmers, you need to stay away from big words like "memory".  I've seen way too many people who struggle with the difference between data in RAM and data on the hard drive - not that virtual memory helps.

    Your best bet will always be a physical metaphor rather than using any other programmer-ese words.  Unfortunately, words like "address" and "pointer" have already been co-opted by earlier languages and now have additional baggage meaning that no longer applies in C#.

    For Larry:

    A different real-world metaphor that might work is the dry-cleaner's claim ticket.  Every object is like a binder notebook, with a piece of information on each page.

    Some notebooks are "value type" that you carry with you, but cannot be shared beyond making copies.

    Other notebooks are "reference type" and are always owned by the GC.  In order to use or modify the information stored in those objects, you have to have the claim ticket information written in your notebook.  You can then use that claim ticket to be allowed access to the referenced notebook, where you can copy or modify the values written there.  Since other notebooks can contain a copy of your claim ticket, owners of those notebooks can also have access to the referenced notebook.

    Fortunately, the C# compiler hides all that extra work needed to use a referenced variable, so you don't have to understand it before you can use it.

    Unfortunately, the C# compiler hides all that extra work needed to use a referenced variable, so you can easily forget the distinction.  Why is there a need for value types, other than optimization?

  • >>>For beginner programmers, you need to stay away from big words like "memory".  I've seen way too many people who struggle with the difference between data in RAM and data on the hard drive - not that virtual memory helps.

    If someone struggles with the difference between data in RAM and data on hard drive then probably he is not a programmer (by any definition).

    I found the the article entertaining and author was able to clearly able to make distinctions between pointer and references. But for the oversimiplication of address is bettwe when it comes to understand how the references can be used in the programs. "Reference is address to the object" is simple enoguh to understand. To make it clear it can be added "But unlike pointer it doesn't allow you to do adress manipulation". OR how about "reference is its Readonly Address?"

  • Sorry for repeat posting...something didn't work properly last time....

    >>>For beginner programmers, you need to stay away from big words like "memory".  I've seen way too many people who struggle with the difference between data in RAM and data on the hard drive - not that virtual memory helps.

    If someone struggles with the difference between data in RAM and data on hard drive then probably he/she is not a programmer (by any definition).

    I found the the article entertaining and author was clearly able to make distinctions between pointer and references. But for me the oversimiplication of address is better when it comes to understand how  references can be used in the programs. "Reference is address to the object" is simple enoguh to understand. To make it clear it can be added "But unlike pointer it doesn't allow adress manipulation". OR how about "reference is **Readonly** Address of an object?"

  • You can understand how the confusion comes around though. Checkout the .NET 2.0 Foundation MCTS training kit which is obviously created by or endorsed by Microsoft (has it's logo all over it) and under a big title:

    What Is a Reference Type: Reference types store the address of their data, also known as a pointer, on the stack.

    If we're taught this in the beginning by the company who develops the language, you can see why the confusion comes around!

  • This is a very interesting article, although I must respectfully disagree with the author on whether novice programmers should hear the word "address" or "memory" without knowing something about the memory model of the language.

    There are few concepts in programming more fundamental than a variable, and I can't think of a language in which a variable is anything but some place in memory holding a value. I also can't think of a language I *must* know in order to understand that concept, whether its in writing a chunk of VBA to automate Excel or delclaring an integer in ol' F77. if you aspire to program, but can't be expected to understand the concept of a variable representing an address, one might want to consider a different profession.

  • >> if you aspire to program, but can't be expected to understand the concept of a variable representing an address, one might want to consider a different profession.

    Except that there's a big difference between writing a program and being in the programming profession.  There are plenty of people out there who are not programmers, but are able to write a little bit of code here and there to get things done.  I'm thinking of office workers writing VBA Excel macros, primarily graphics oriented web developers writing a small amount of glue code or JavaScript, or any other kind of hobbyist programmer.  These are people who have no interest in taking CS101, nor should they be expected to.

  • David W Wrote>

    "if you aspire to program, but can't be expected to understand the concept of a variable representing an address, one might want to consider a different profession."

    Totally agree with you!!!....

    New programmers are happy codifying in this "abstract" or "opaque" concept. Like some says above..."you arrives to the store... and choose any product stored in it"...

    Maybe the use of delegates clarify more this concept: references vs pointers....

    Excelent post and useful and intersting point of views... thank you partners....!!

  • As for me I can treat "reference" as functor that encapsulates object identity (in notation of Grady Booch) and this case it is not possible to think about equality between reference and pointer (first of all because possible difference of types of pointer on object and object identifier that can be complex in general).

    Moreover if we consider STL where we shouldn't retrieve and use pointer on objects stored in collection. While identifier is a constant for an instance, the pointer as physical address can be changed at any time by memory manager. Similar approach was used in Windows API where functions use handlers for all API objects.

    Thus reference is a pure proxy of an object. And at run time object itself can be created at another time as it is in one of use cases of GoF pattern "Proxy".

    So, pointer as an address very useful when we aware of immutability of object physical address - during execution of member function. Here we can try to obtain pointer of "this" and to directly manipulate on object aggregated by value objects. It makes possible to create more efficient code that doesn't execute unnecessary verification and access to members using mathematically calculated pointers. I think that it is a main reason to have 2 types of complex objects: reference types and value types.

  • I am a Teacher, and I usualy explain my student at first that a reference type is crreated on the heap and the address of the place where it was created will be saved on the stack. Then I make sure that they understand that they can not manipulate this address in any way, because garbage collector is owning this address, and can move it in order to compact the memory. At this stage in their career doesn't metter for them the unsafe code part of the .NET. Later on, when they will get some more experience, they will get the difference anyway.

  • c,c++对于一个初学计算机的人来说有一定的难度,希望Microsoft有关于visual basic方面的教程.

  • For novices, "a reference is a address of a variable in process space" is a good & quick way to understanding a reference, although the address is not static in C#.

Page 4 of 6 (76 items) «23456