Knights, Knaves, Protected and Internal

Knights, Knaves, Protected and Internal

Rate This
  • Comments 42

Knight When you override a virtual method in C# you are required to ensure that the stated accessibility of the overridden method - that is, whether it is public, internal, protected or protected internal(*) – is exactly re-stated in the overriding method. Except in one case. I refer you to section 10.6.4 of the specification, which states:

an override declaration cannot change the accessibility of the virtual method. However, if the overridden base method is protected internal and it is declared in a different assembly than the assembly containing the override method then the override method’s declared accessibility must be protected.

What the heck is up with that? Surely if an overridden method is protected internal then it only makes sense that the overriding method should be exactly the same: protected internal.

I’ll explain why we have this rule, but first, a brief digression.

A certain island is inhabited by only knights and knaves. Knights make only true statements and only answer questions truthfully; knaves make only false statements and only answer questions untruthfully. If you walk up to an inhabitant of the (aptly-named) Island of Knights and Knaves you can rapidly ascertain whether a particular individual is a knight or a knave by asking a question you know the answer to. For example “does two plus two equal four?” A knight will answer “yes” (**), and a knave will answer “no”. Knaves are prone to saying things like “my mother is a male knight”, which is plainly false.

It might seem at first glance that there is no statement which could be made by both a knight and a knave. Since knights tell the truth and knaves lie, they cannot both make the same statement, right? But in fact there are many statements that can be made by both. Can you think of one?

.

.

.

.

.

.

.

Both a knight and a knave can say “I am a knight.”

How does that work? The reason this works is because the pronoun “I” refers to different people when uttered by different people. If Alice, a knight, makes the statement “I am a knight”, she is asserting the truth that “Alice is a knight”. If Bob, a knave, makes the statement “I am a knight”, he is not asserting the true statement “Alice is a knight” but rather the false statement “Bob is a knight”. Similarly, both Alice and Bob can assert the statement "My name is Alice," for the same reason.

And that’s why overriding methods in a different assembly aren’t “protected internal”. The modifier “internal” is like a pronoun; it refers to the current assembly. When used in two different assemblies it means different things. The purpose of the rule is to ensure that a derived class does not make the accessibility domain of the virtual member any larger or smaller.

An analogy might help. Suppose a protected resource is a car, an assembly is a dwelling, a person is a class and descendent is a derived class.

  • Alice has a Mazda and lives in House with her good friend Charlie.
  • Charlie has a child, Diana, who lives in Apartment.
  • Alice has a child, Elroy, who lives in Condo with his good friend Greg.
  • Elroy has a child – Alice’s grandchild -- Frank, who lives in Yurt.

Alice grants access to Mazda to anyone living in House and any descendent of Alice. The people who can access Mazda are Alice, Charlie, Elroy, and Frank.

Diana does not get access to Mazda because she is not Alice, not a descendent of Alice, and not a resident of House. That she is a child of Alice’s housemate is irrelevant.

Greg does not get access to Mazda for the same reason: he is not Alice, not a descendent of Alice, and not a resident of House. That he is a housemate of a descendent of Alice is irrelevant. 

Now we come to the crux of the matter. Elroy is not allowed to extend his access to Mazda to Greg. Alice owns that Mazda and she said "myself, my descendents and my housemates". Her children don't have the right to extend the accessibility of Mazda beyond what she initially set up. Nor may Elroy deny access to Frank; as a descendent of Alice, Frank has a right to borrow the car and Greg cannot stop him by making it "private".

When Elroy describes what access he has to Mazda he is only allowed to say "I grant access to this to myself and my descendents" because that is what Alice already allowed. He cannot say "I grant access to Mazda to myself, my descendents and to the other residents of Condo".

-----------------

(*) Private virtual methods are illegal in C#, which irks me to no end. I would totally use that feature if we had it.

(**) Assuming that they answer at all, of course; there could be mute knights and knaves.

  • Yes, why are private virtual. methods illegal?  I know of some experts in the C++ communithy that assert that all vritual methods should be private!  

    Another irksome thing is that overrides can't make a member more accessible.  There certainly are reasonable designs where a member that was an internal detail in a base class is a sensible part of the public API in a derived class.  Making overrides more visible doesn't violate LSP or break any important scenarios that I can think of, so why is this not allowed in C#?

  • I sometimes wish that the accessibility of the overriding method could be either wider or narrower than the accessibility of the overridden method. Then I could avoid creating (and coming up with a name for) a forwarding method when I want to change the accessibility of either an overriding or an overridden method from protected to public.

  • There's a much simpler explanation.

    Making an external override protected internal would make it internal to the overriding assembly, which (outside the inheriting type) has no business seeing the method.  I don't see why you need the whole analogy.

    I think you'll find that in general it is rather easy to explain things to people who already understand in depth what you're attempting to explain. Successfully explaining things to people who don't already have a deep understanding of the topic is rather more difficult. -- Eric 

  • Is that a nitpicker's corner developing at the bottom there? ;)

    I do wish you could sometimes make the scope narrower on overridden methods.  Sometimes I find myself with a hierarchy two or three levels deep where I want to turn off one little bit of functionality for any derived classes and I have to end up throwing a bool in and checking for that feature.  I'm not terribly sure how you'd implement this so callers expecting a certain visibility didn't get shafted when something came by that hid that method.

  • @Carl D:

    Making a method more accessible could end up allowing a caller to break the invariants of a class by having it called in unexpected ways or at unexpected times.  For example, suppose there is a BankAccount class with a protected internal method SetBalance.  The protected portion of the modifier communicates that derived classes are trusted to

        1. Alter the default behavior of setting the balance as necessary (override)

        2. Decide when the balance can be set (call)

    The internal portion of the modifier communicates that classes in the same assembly as BankAccount should be able to call the SetBalance method as necessary.  Obviously, even these trusted classes should not be able to alter the behavior of the SetBalance method.

    If a derived class were able to declare the method more accessible, it would be possible to allow untrusted callers (outside of BankAccount's assembly) to call the method, potentially altering the balance in unforeseen ways. Derived classes are already able to do the same thing, which is a strong argument for private virtual methods.

  • The problem is that 'protected' and 'internal' are both restrictive terms in English. So someone applying both of them may naively assume that _both restrictions_ (relative to public) apply, rather than that _both relaxations_ (relative to private) apply. Both concepts exist in the CLR, but "FamilyAndAssembly" access isn't available in C#.

    "Elroy is not allowed to extend his access to Mazda to Greg"

    It sounds reasonable enough in your carefully-chosen analogy, but this is a principle that is not and cannot be enforced by the type system: public Car GetMazda() { return Mazda; } - by making a member protected you _are_ giving permission for any derived class to give access to it to anyone if they choose to.

    As for "private virtual" methods - or, more precisely, private override methods - C# actually has those in the form of explicit interface implementations. It's unclear why there can't be explicit base class override implementations as well. Keeping in mind that they would become accessible by casting to the base class.

    What would be the point of a private virtual method if it _isn't_ narrowed from a more accessible method it overrides from the base class?

  • OK, I'm lost on this whole "private virtual" thing. Tell me where I'm wrong here: A "private" method is a method that only the base class knows about; a derived class cannot call that method and can create its own method by that name without needing "new" to avoid a warning. A "virtual" method is a method which has an implementation in the base class, but for which a derived class can provide an overriding implementation if it wants to. Logically, a "private virtual" method would be one which the derived class can override but not know about. How does that make sense?

    You've forgotten that the derived class can be a nested class; nested classes have access to the private methods of their containers. I quite like the pattern of making an abstract base class with a private constructor so that the only concrete classes possible are those declared within the base class itself. But if you want to create a virtual method that only precisely those derived classes you know about can override, the best you can do is make it internal. -- Eric

    I'm going to guess that people actually want something else, like a method that can be overridded in a derived class but hidden from children of the derived classes. This would entail a method being visible to a class's parent but not said class's children. If that's the case, maybe what you want is more like "semiprivate virtual" or "overprotected virtual" or use a different definition of "virtual" from what I understand it to be.

    As for external assemblies not being able to declare functions "override internal protected", why not? It seems like I can easily create "override protected Car GetMazda() { return Mazda; }" and then "internal Car ShadowGetMazda() { return Mazda; }", so why force me to pollute my class's namespace like that?

  • Knights and Knaves can both say, "The time is 12:15". I don't believe this has any relevance.

  • Ah, so "protected internal" means protected "union" internal, not protected "intersection" internal.

    <i>Private virtual methods are illegal in C#, which irks me to no end. I would totally use that feature if we had it.</i>

    Why are they illegal? IOW, what were the language designers thinking? Would it break existing code to make them legal?

  • > Making an external override protected internal would make it internal to the overriding assembly, which (outside the inheriting type) has no business seeing the method.

    But the assembly can already expose a protected internal method as internal, by wrapping it in a method declared as internal. It just requires one extra step of indirection from the guy who's doing it in his class, but the result is precisely the same.

    There is an old adage regarding "protected" modifier, which goes as follows: "Remember that 'protected' really means 'public'!". And that is very true. Sure, you only expose the member to derived types - but they can expose it to anyone they want! The only exception is when you control the entire hierarchy by e.g. making the constructor private or internal.

    In light of that, what is the purpose of restriction of widening visibility on override? Just an extra hoop to jump through to ensure that whoever is doing it, really know what they want?

  • > OK, I'm lost on this whole "private virtual" thing. Tell me where I'm wrong here: A "private" method is a method that only the base class knows about; a derived class cannot call that method and can create its own method by that name without needing "new" to avoid a warning. A "virtual" method is a method which has an implementation in the base class, but for which a derived class can provide an overriding implementation if it wants to. Logically, a "private virtual" method would be one which the derived class can override but not know about. How does that make sense?

    Private virtual methods exist in C++, and there is a school of thought (to which I subscribe) which advocates their use in lieu of protected virtual methods.

    The problem with understanding that you have comes from overly narrow definition of "private". At least in C++, "private" doesn't mean "something that only the class knows about". It means "something that can only be referenced by name from within that class". There are various consequences for this rule - for example, in C++, a public class method can return a private struct declared within the type, and the method would still be callable from outside; furthermore, members of the struct it returns would also be accessible, so long as they themselves are public. You can pass the returned value to a function template using template argument inference, and the private type will be inferred; from there, you actually have an alias to it, and can declare variables of that type etc. Or, in C++0x, you can just use auto/decltype.

    The important consequence, though, is that, while a private virtual method cannot be called from outside the class, it can be overridden in classes derived from that class, since an overriding method does not in any way references the method it overrides by name; it just happens to be a method with the same name, and that is it.

    The reason why you'd want this is when virtual methods are used strictly as hooks (e.g. like Collection<T>.AddItem in .NET). In this case, they are provided solely for derived classes to override, but they already have a strict contract on when, exactly, they will be called. If they are protected, derived classes can break that contract by calling them themselves. By making them private, you ensure that only the base class can call them, and thus guarantee that contract always holds, for the entire hierarchy (i.e. you don't have to trust any intermediate base classes in the chain to not break things for you).

    One could argue that "private" is, perhaps, a misnomer for it in general. Nonetheless, due to the peculiar definition of "private" in C++, and the established use of "private virtual" in C++ as a consequence of that, it is what it is.

  • > Why are they illegal? IOW, what were the language designers thinking? Would it break existing code to make them legal?

    I suspect that there is some simple rule of thumb, such as "private members of the class are never part of its API surface", that they wanted to preserve. Private virtual methods aren't really private in that sense - now you have to document them, all class versioning problems apply to them, etc.

  • "When Elroy describes what access he has to Mazda he is only allowed to say "I grant access to this to myself and my descendants" "

    But if Elroy decides to replace the Mazda with a newer faster Mazda 3 (ie, override the Mazda), he may be doing so with the intention (based on his declaration) that only his relatives (descendants by virtue of 'protected' and ancestors/siblings by virtue of 'virtual override') can use it.  But little does he know, Charlie can actually use his brand new M3, because Charlie just thinks it's Alice's car...

    I can't see any situation where this could really be a problem, because Elroy has no way of providing visibility of anything back to Alice without also providing visibility to Charlie, but is this actually the case, or is there something else that could happen?

    PS: Where's Bob?  Why'd Eve get a gender reassignment?

  • Chris B.:

    > Making a method more accessible could end up allowing a caller to break the invariants of a class by having it called in unexpected ways or at unexpected times. [...]

    I don't see the logic in this.  If a method is declared protected, then a subclass is able to call that method at will, or to forward another (more accessible) method call to it.  Disallowing making an overridden method more accessible does nothing to maintain invariants.  The subclass can break invariants in any number of other ways if it chooses to do so.

    I understand why narrowing accessibility is disallowed, but I still fail to understand the reasoning behind disallowing widening accessibility.  If A.f() is protected, and B.f() overrides A.f() and makes the method public, B has done nothing that it couldn't do by simply exposing B.g() publicly and forwarding that to A.f().  Or to put it in terms of Elroy and Greg, it seems that Elroy can easily give Greg access to Mazda.  They just call it Hyundae instead.

  • @Derek: indeed, the CLR permits increasing the visibility of overridden methods. It's C# that's lacking.

Page 1 of 3 (42 items) 123