August, 2009

  • The Old New Thing

    Why can't I declare a type that derives from a generic type parameter?

    • 18 Comments

    A lot of questions about C# generics come from the starting point that they are just a cutesy C# name for C++ templates. But while the two may look similar in the source code, they are actually quite different.

    C++ templates are macros on steroids. No code gets generated when a template is "compiled"; the compiler merely hangs onto the source code, and when you finally instantiate it, the actual type is inserted and code generation takes place.

    // C++ template
    template<class T>
    class Abel
    {
    public:
     int DoBloober(T t, int i) { return t.Bloober(i); }
    };
    

    This is a perfectly legal (if strange) C++ template class. But when the compiler encounters this template, there are a whole bunch of things left unknown. What is the return type of T::Bloober? Can it be converted to an int? Is T::Bloober a static method? An instance method? A virtual instance method? A method on a virtual base class? What is the calling convention? Does T::Bloober take an int argument? Or maybe it's a double? Even stranger, it might accept a Canoe which gets converted from an int by a converting constructor. Or maybe it's a function that takes two parameters, but the second parameter has a default value.

    Nobody knows the answers to these questions, not even the compiler. It's only when you decide to instantiate the template

    Abel<Baker> abel;
    

    that these burning questions can be answered, overloaded operators can be resolved, conversion operators can be hunted down, parameters can get pushed on the stack in the correct order, and the correct type of call instruction can be generated.

    In fact, the compiler doesn't even care whether or not Baker has a Bloober method, as long as you never call Abel<Baker>::DoBloober!

    void f()
    {
     Abel<int> a; // no error!
    }
    
    void g()
    {
     Abel<int> a;
     a.DoBloober(0, 1); // error here
    }
    
    Only if you actually call the method does the compiler start looking for how it can generate code for the DoBloober method.

    C# generics aren't like that.

    Unlike C++, where a non-instantiated template exists only in the imaginary world of potential code that could exist but doesn't, a C# generic results in code being generated, but with placeholders where the type parameter should be inserted.

    This is why you can use generics implemented in another assembly, even without the source code to that generic. This is why a generic can be recompiled without having to recompile all the assemblies that use that generic. The code for the generic is generated when the generic is compiled. By comparison no code is generated for C++ templates until the template is instantiated.

    What this means for C# generics is that if you want to do something with your type parameter, it has to be something that the compiler can figure out how to do without knowing what T is. Let's look at the example that generated today's question.

    class Foo<T>
    {
     class Bar : T
     { ... }
    }
    

    This is flagged as an error by the compiler:

    error CS0689: Cannot derive from 'T' because it is a type parameter
    

    Deriving from a generic type parameter is explicitly forbidden by 25.1.1 of the C# language specification. Consider:

    class Foo<T>
    {
     class Bar : T
     {
       public void FlubberMe()
       {
         Flubber(0);
       }
     }
    }
    

    The compiler doesn't have enough information to generate the IL for the FlubberMe method. One possibility is

    ldarg.0        // "this"
    ldc.i4.0    // integer 0 - is this right?
    call T.Flubber // is this the right type of call?
    

    The line ldc.i4.0 is a guess. If the method T.Flubber were actually void Flubber(long l), then the line would have to be ldc.i4.0; conv.i8 to load an 8-byte integer onto the stack instead of a 4-byte integer. Or perhaps it's void Flubber(object o), in which case the zero needs to be boxed.

    And what about that call instruction? Should it be a call or callvirt?

    And what if the method returned a value, say, string Flubber(int i)? Now the compiler also has to generate code to discard the return value from the top of the stack.

    Since the source code for a generic is not included in the assembly, all these questions have to be answered at the time the generic is compiled. Besides, you can write a generic in Managed C++ and use it from VB.NET. Even saving the source code won't be much help if the generic was implemented in a language you don't have the compiler for!

  • The Old New Thing

    Why doesn't String.Format throw a FormatException if you pass too many parameters?

    • 19 Comments

    Welcome to CLR Week 2009. As always, we start with a warm-up.

    The String.Format method doesn't throw a FormatException if you pass too many parameters, but it does if you pass too few. Why the asymmetry?

    Well, this is the type of asymmetry you see in the world a lot. You need a ticket for each person that attends a concert. If you have too few tickets, they won't let you in. If you have too many, well, that's a bit wasteful, but you can still get in; the extras are ignored. If you create an array with 10 elements and use only the first five, nobody is going to raise an ArrayBiggerThanNecessary exception. Similarly, the String.Format message doesn't mind if you pass too many parameters; it just ignores the extras. There's nothing harmful about it, just a bit wasteful.

    Besides, you probably don't want this to be an error:

    if (verbose) {
      format = "{0} is not {1} (because of {2})";
    } else {
      format = "{0} not {1}";
    }
    String.Format(format, "Zero", "One", "Two");
    

    Think of the format string as a SELECT clause from the dataset provided by the remaining parameters. If your table has fields ID and NAME and you select just the ID, there's nothing wrong with that. But if you ask for DATE, then you have an error.

  • The Old New Thing

    Actually, FlagsAttribute can't do more; that's why it's an attribute

    • 27 Comments

    A few years ago, Abhinaba wondered why FlagsAttribute didn't also alter the way enumeration values are auto-assigned.

    Because attributes don't change the language. They are instructions to the runtime environment or (in rarer cases) to the compiler. An attribute can instruct the runtime environment to treat the function or class in a particular way. For example, you can use an attribute to tell the runtime environment that you want the program entry point to run in a single-threaded apartment, to tell the runtime environment how to look up your p/invoke function, or to tell the compiler to suppress a particular class of warnings.

    But changing how values for enumerations are assigned, well that actually changes the language. An attribute can't change the operator precedence tables. An attribute can't change the way overloaded functions are resolved. An attribute can't change the statement block tokens from curly braces to square braces. An attribute can't change the IL that gets generated. The code still compiles to the same IL; the attribute just controls the execution environment, such as how the JIT compiler chooses to lay out a structure in memory.

    Attribute or not, enumerations follow the same rule for automatic assignment: An enumeration symbol receives the value one greater than the previous enumeration symbol.

  • The Old New Thing

    Why can't I pass a reference to a derived class to a function that takes a reference to a base class by reference?

    • 23 Comments

    "Why can't I pass a reference to a derived class to a function that takes a reference to a base class by reference?" That's a confusing question, but it's phrased that way because the simpler phrasing is wrong!

    Ths misleading simplified phrasing of the question is "Why can't I pass a reference to a derived class to a function that takes a base class by reference?" And in fact the answer is "You can!"

    class Base { }
    class Derived : Base { }
    
    class Program {
      static void f(Base b) { }
    
      public static void Main()
      {
          Derived d = new Derived();
          f(d);
      }
    }
    

    Our call to f passes a reference to the derived class to a function that takes a reference to the base class. This is perfectly fine.

    When people ask this question, they are typically wondering about passing a reference to the base class by reference. There is a double indirection here. You are passing a reference to a variable, and the variable is a reference to the base class. And it is this double reference that causes the problem.

    class Base { }
    class Derived : Base { }
    
    class Program {
      static void f(ref Base b) { }
    
      public static void Main()
      {
          Derived d = new Derived();
          f(ref d); // error
      }
    }
    

    Adding the ref keyword to the parameter results in a compiler error:

    error CS1503: Argument '1': cannot convert from 'ref Derived' to 'ref Base'
    

    The reason this is disallowed is that it would allow you to violate the type system. Consider:

      static void f(ref Base b) { b = new Base(); }
    

    Now things get interesting. Your call to f(ref d) passes a reference to a Derived by reference. When the f function modifies its formal parameter b, it's actually modifying your variable d. What's worse, it's putting a Base in it! When f returns, your variable d, which is declared as being a reference to a Derived is actually a reference to the base class Base.

    At this point everything falls apart. Your program calls some method like d.OnlyInDerived(), and the CLR ends up executing a method on an object that doesn't even support that method.

    You actually knew this; you just didn't know it. Let's start from the easier cases and work up. First, passing a reference into a function:

    void f(SomeClass s);
    
    ...
       T t = new T();
       f(t);
    

    The function f expects to receive a reference to a SomeClass, but you're passing a reference to a T. When is this legal?

    "Duh. T must be SomeClass or a class derived from SomeClass."

    What's good for the goose is good for the gander. When you pass a parameter as ref, it not only goes into the method, but it also comes out. (Not strictly true but close enough.) You can think of it as a bidirectional parameter to the function call. Therefore, the rule "If a function expects a reference to a class, you must provide a reference to that class or a derived class" applies in both directions. When the parameter goes in, you must provide a reference to that class or a derived class. And when the parameter comes out, it also must be a reference to that class or a derived class (because the function is "passing the parameter" back to you, the caller).

    But the only time that S can be T or a subclass, while simultaneously having T be S or a subclass is when S and T are the same thing. This is just the law of antisymmetry for partially-ordered sets: "if a ≤ b and b ≤ a, then a = b."

  • The Old New Thing

    Common gotchas when writing your own p/invoke

    • 21 Comments

    If you're looking to get into some p/invoke action, you'd be well-served to check out the pinvoke wiki to see if somebody else has done it too. If what you need isn't there, you may end up forced to write your own, and here are some gotchas I've seen people run into:

    • C++ bool and Win32 BOOLEAN are not the same as C# bool (aka System.Boolean). In Win32, BOOL is a 4-byte type, and BOOLEAN is a 1-byte type. [See also MadQ's remarks about VARIANT_BOOL.] Meanwhile, C++ bool is not standardized by Win32, so the size will vary based on your compiler, but most compilers use a 1-byte value. And then C# is even weirder: The bool is a 1-byte type, but it marshals as a 4-byte type by default.
    • Win32 char is not the same as C# char (aka System.Char). In C#, char is a Unicode character (two bytes), whereas in C/C++ under Win32 it is an ANSI character (one byte).
    • Win32 long is not the same as C# long (aka System.Int64). In C#, long is 64-bit value, whereas in C/C++ under Win32 it is a 32-bit value.
    • If memory is allocated and freed across the interop boundary, make sure both sides are using the same allocator. It is my understanding that the CLR uses CoTaskMemAlloc/CoTaskMemFree by default. If your Win32 function doesn't use CoTaskMemAlloc, you'll have to teach the CLR which allocator you really want.
    • When laying out structures, you have to watch out for alignment.

    That last one is particularly gnarly on 64-bit systems, where alignment requirements are less forgiving than on x86. The structure declarations on pinvoke.net tend to ignore 64-bit issues. For example, the declaration of the INPUT structure (as of this writing—it's a wiki so it's probably changed by the time you read this) reads as follows:

    [StructLayout(LayoutKind.Explicit)]struct INPUT {
      [FieldOffset(0)] int type;
      [FieldOffset(4)] MOUSEINPUT mi;
      [FieldOffset(4)] KEYBDINPUT ki;
      [FieldOffset(4)] HARDWAREINPUT hi;
    }
    

    This structure layout is correct for 32-bit Windows, but it's incorrect for 64-bit Windows.

    Let's take a look at that MOUSEINPUT structure, for starters.

    typedef struct tagMOUSEINPUT {
        LONG    dx;
        LONG    dy;
        DWORD   mouseData;
        DWORD   dwFlags;
        DWORD   time;
        ULONG_PTR dwExtraInfo;
    } MOUSEINPUT, *PMOUSEINPUT, FAR* LPMOUSEINPUT;
    

    In 64-bit Windows, the LONG and DWORD members are four bytes, but the dwExtraInfo is a ULONG_PTR, which is eight bytes on a 64-bit machine. Since Windows assumes /Zp8 packing, the dwExtraInfo must be aligned on an 8-byte boundary, which forces four bytes of padding to be inserted after the time to get the dwExtraInfo to align properly. And in order for all this to work, the MOUSEINPUT structure itself must be 8-byte aligned.

    Now let's look at that INPUT structure again. Since the MOUSEINPUT comes after the type, there also needs to be padding between the type and the MOUSEINPUT to get the MOUSEINPUT back to an 8-byte boundary. In other words, the offset of mi in the INPUT structure is 8 on 64-bit Windows, not 4.

    Here's how I would've written it:

    // This generates the anonymous union
    [StructLayout(LayoutKind.Explicit)] struct INPUT_UNION {
      [FieldOffset(0)] MOUSEINPUT mi;
      [FieldOffset(0)] KEYBDINPUT ki;
      [FieldOffset(0)] HARDWAREINPUT hi;
    };
    
    [StructLayout(LayoutKind.Sequential)] struct INPUT {
      int type;
      INPUT_UNION u;
    }
    

    I introduce a helper structure to represent the anonymous union that is the second half of the Win32 INPUT structure. By doing it this way, I let somebody else worry about the alignment, and it'll be correct for both 32-bit and 64-bit Windows.

    static public void Main()
    {
      Console.WriteLine(Marshal.OffsetOf(typeof(INPUT), "u"));
    }
    

    On a 32-bit system, this prints 4, and on a 64-bit system, it prints 8. The downside is that you have to type an extra u. when you access the mi, ki or hi members.

    input i;
    i.u.mi.dx = 0;
    

    (I haven't checked what the PInvoke Interop Assistant comes up with for the INPUT structure.)

  • The Old New Thing

    Why does Windows wait longer than my screen saver idle timeout before starting the screen saver?

    • 32 Comments

    You may find yourself in a situation where Windows takes longer than your specified screen saver idle timeout to start the screen saver.

    First of all, there are ways for programs to block the screen saver entirely. Calling SetThreadExecutionState(ES_DISPLAY_REQUIRED), is how a program says, "Even though there is no mouse or keyboard input, the screen is indeed in use, so don't blank it or start the screen saver." Media playback programs use this so the screen saver doesn't kick in while you're watching a movie on your DVD, and presentation programs use it so the screen saver doesn't start in the middle of your multi-million-dollar proposal.

    But even if no program has marked the screen as busy, Windows itself will delay activating the screen saver if it detects that you would prefer that it not run for a while.

    If you repeatedly dismiss the screen saver less than one minute after it starts on Windows Vista or later, the operating system says, "Oh, sorry. I thought there was nobody there, but obviously there is. You're probably reading an information-dense document or using your laptop as a flashlight or clock, and you want the screen to stay on even though you aren't generating any input. I'll hold off the screen saver for a little while for you."

    After the second time you do a "fast dismiss" of the screen saver, the screen saver idle time is temporarily incremented by its nominal value. For example, if you set your screen saver timeout to two minutes, then starting with the second fast dismiss, Windows will wait an additional two minutes before trying the screen saver again. Here's a timeline for people who like timelines:

    • At T = 0, you stop generating input.
    • At T = 2m, the screen saver starts. You dismiss it immediately.
    • At T = 4m, the screen saver starts. You dismiss it immediately. This is your second consecutive fast dismiss, so the screen saver timeout is temporarily increased to 4 minutes.
    • At T = 8m, the screen saver starts. You dismiss it immediately. This is your third consecutive fast dismiss, so the screen saver timeout is temporarily increased to 6 minutes.
    • At T = 14m, the screen saver starts. You dismiss it immediately. This is your fourth consecutive fast dismiss, so the screen saver timeout is temporarily increased to 8 minutes.

    As long as you keep fast-dismissing, the delay will increase. Of course, the delay won't grow indefinitely, because you'll eventually hit some other idle timeout, like the system sleep timeout, and then the system will sleep before it gets a chance to run the screen saver.

    If this feature offends you, then you can disable it by setting Adaptive Display to off in the Advanced Power Settings. There are also Group Policies for controlling this feature.

  • The Old New Thing

    Programming means that sometimes you have to snap two blocks together

    • 53 Comments

    Part of the challenge of programming (and for some people, the reason why programming is fun in the first place) is looking at the building blocks provided to you and deciding how to assemble them to build something new. After all, if everything you wanted a program to do already existed ready-made, it wouldn't be called programming any more. It would be called shopping.

    Is there an API or a quick way to find out which window the mouse is in?

    I replied, "The LEGO Group does not make a piece for every possible object. Sometimes you just have to take two LEGO blocks and click them together. Here are some interesting blocks: GetCursorPos, WindowFromPoint."

    Thanks for your reply. But WindowFromPoint gives me the window of the object at the location of the cursor. But I'm looking for the top level window containing the cursor.

    Fine, then use a different block.

    I wonder how it is these people manage to write programs at all. I get the impression they write code by asking a million tiny questions and cutting and pasting together all the replies.

    No wait, pasting together the replies counts as snapping blocks together. Maybe they just ask for completed programs.

  • The Old New Thing

    What is the maximum number of timers a program can create?

    • 12 Comments

    As MSDN correctly notes, "Timers are a limited global resource." How limited are they? (We're talking about timers created by SetTimer.)

    Not as limited as they once were.

    Originally, there were eight timers total for the entire system. When there are only eight timers, you have to be very careful not to waste them and to destroy them when they aren't needed.

    Windows 3.0 increased this to 32.

    Windows 95 increased this to around 2500, although the debug version of Windows got mad at you if a single program created more than 32.

    Windows NT 3.1 would create timers until you exhausted the desktop heap. The maximum number of timers depends on what other window manager objects had been created.

    Starting with Windows NT 4, the window manager treats timers as User Objects for bookkeeping purposes, so all your timers, windows, menus, and so on fight for space in the per-process 10,000 object limit and the 32,000-ish object limit for a desktop.

    As you can see, the precise limit changes over time, so you couldn't count on any particular limit being in effect. Who knows, maybe the limit will change again in a future version of Windows.

    These limits are pretty high, but that doesn't mean you can be lazy. Jessica pointed out that the WinForms Timer component is easily leaked "because most people don't know that they have to call Dispose on it." Another thing to check for when you find that you can't create any more timers.

  • The Old New Thing

    The wisdom of seve^H^H^H^Hsixth graders: What it means to be an adult

    • 24 Comments

    I was out of town for the grading of the seventh grade essays, so I pitched in with the sixth grade essays instead. The students were asked to think of an adult and describe the qualities that make that person an adult. This topic was not very well received by the students, who deemed it uncreative and boring. While I understand their lack of enthusiasm, it's also true that for most of your life, you're going to have to write on topics that are uncreative and boring (and the stakes are going to be higher), so you'd better get good at it.

    The difference in writing skill between sixth and seventh graders (between eleven year olds and twelve year olds) is quite noticeable. Many of sixth graders could not get past the literal definition of the word adult, describing the qualities that make an adult purely in terms of biology: Age, height, strength, puberty, armpit hair. Many others focused on accomplishments or privileges that distinguish adults from children: Advanced education, having a job, knowing how to drive a car, and being able to stay up late without getting yelled at.

    Remember, these are just the funny sentences/excerpts. Do not assume that all students write like this. The assignment is given under standardized test conditions: 90 minutes with nothing but pencil and paper, with one additional hour available upon request.

    The easy life

    • After collage, you could just go home and relax for the rest of your life. And you wonder why scrapbooking is so popular? It's because everybody goes to collage!
    • Adults go on vacation to international countries and play golf. Gosh, I wonder if this student comes from a wealthy family.
    • My dad is a big fan of football. Who's not?

    Check your fun at the door

    • Adults don't like to do anything fun 60% of the time. I think I'm getting shortchanged on the other 40%, too.
    • They talk and talk and talk, that's all they do.
    • When an adult takes you somewhere it is usually to a depactment store.
    • My mom was so busy she had to step up her gear to get it all done.
    • My mom is nice, she likes to get new kitchen supplies and carpet.
    • Just like kids, adults still make mistakes and just want to have fun.
    • Being an adult means living above the influence.
    • Adults don't do stupid things like throwing wild house parties 24/7.
    • Mature people order off the adult menu at Red Robin and do not order jumbo sundaes with extra cherries.

    Responsible behavior

    • My mom is responsible because she cleans the house before anybody tells her to.
    • My dad is good sport. If he wins something, he doesn't say, "nanny nanny boo boo."
    • Being mature is one big part about being an adult because if no one was mature then we would be at war all the time. Instead, adults just talk and talk and talk.
    • My mom cleans the house because my dad doesn't. Are you suggesting that your dad isn't an adult?
    • Being a civilized adult is simple, and all adults have it just not all the time.

    Let me tell you about my parents

    • She's now a mail women.
    • My dad is an adult because of his hair. He is losing his hair, and I hope he will not be bald soon.
    • He has a wife (my mom). Thanks for clearing that up.
    • My dad is really smart. He's been married three times. Most people are satisfied to be only one third as smart as him.
    • A good dad makes his own meals, and who doesn't like a guy that knows a thing or two about the stove?
    • My dad is tough. He is not easily scared by spiders, lightening, or the dark. I'm assuming that lightening was a spelling error, but who knows?
    • My mom helps me appreciate that I don't live in a third world country. For example, she frequently reminds me that there are starving children in Africa.
    • I hope one day my mom will live longer than ever before. "Hey Mom, have you ever been this old before?" "How about now?"
    • My mom is the first adult I ever met.
    • My parents are not overly protective. They let me eat raw cookie dough.
    • When I saw my Uncle Mike in Texas, I knew he was an adult.
    • Sarah has many personalities that make her an adult.

    Assorted commentary

    • Adults can't whine unless their car gets totaled, or their house burns down.
    • But I think of you more as an adult when you have to shave your back hair.
    • My dad is one of the most manly adults I know. Must be the back hair.
    • Eight-teen is a huge age!
    • OK, you've survived the first paragraph. Good, because here comes the second one.
    • Most adults I know are pretty smart. Not Albert Einstein smart, but common sense smart.
    • Adults are great people unless they aren't what they shouldn't be.
    • Every single adult has gone through puberty or at least half way. I believe the ones that went through only halfway are known as 'Frat boys'.
    • Best opening sentence: When you grow up to be an adult, you get armpit hair.

    Concluding thoughts

    • I also think I'm going to have a hard time when I be an adult.
    • And those are just some snidbits from my brain.

    Misspelling corner. I've included more context; that may make the game a bit easier.

    • He doesn't goof around when he is so post to be doing work.
    • Responsibility is when you have to make the right desigin.
    • Being grown up is never goffing up.
    • All the school gets the money from us for fiead hips and fun razors and the suplise. I'm trying to imagine what a fun razor is.
    • Less mature human beans are not very keen and when doing extensive work they throw tantrums.
    • They should be aloud to drive. Speak up! I can't hear you driving.
    • When you become an adult you get fatiol hair.
    • She doesn't wine like a baby. Hey, baby, how about another glass of chardonnay?
    • My dad also plays motable instruments.

    Other remarks on student writing:

    • Most essays followed the standard introduction formula "There are three qualities that make a person an adult. Those qualities are A, B, and C." It's a treat to find an essay that opens more creatively, but alas, the essays with excellent introductions failed to maintain the quality level for the rest of the essay.
    • You can't just write "And that's why X" to conclude your paragraph if you never actually explained why X. I read many paragraphs that took the form "Adults have quality Q. Quality Q means that XYZ. That's why adults should have quality Q." The paragraph did nothing to explain why quality Q is an important one for adults to have; it merely stated and defined it. One teacher explained to me that this isn't a conscious writing choice but is rather simply a bad habit. "I've written a bunch on topic X. Now I need to wrap it up. And the way you wrap it up is to write, 'And that's why X.'"

    And that's why I read student essays.

  • The Old New Thing

    The way to stop people from copying files to a folder is to use NTFS security, not to block drag/drop

    • 19 Comments

    A customer wanted to prevent users from copying files to certain locations, and they did it by hooking functions like SHFileOperation and failing the operation if the parameters were not to its liking. The customer found that the hooks stopped working in Windows Vista because Explorer in Windows Vista uses the new IFileOperation COM interface instead of using the old SHFileOperation function. The customer wanted assistance in getting their hook working again so they could prevent users from copying files to directories they wanted to block.

    Well, first of all, arbitrary function hooking is not supported by any version of Windows, so the customer was already in unsupported territory right off the bat. (There are some components which have an infrastructure for hooks, such as file system filter drivers or Winsock Layered Service Providers.)

    Second, attempting to hook SHFileOperation to prevent the user from copying files into specific directories is looking at the problem at the wrong level, similar to the people who want to block drag/drop when what they really want to block is accidental drag/drop. If you block copying files via drag/drop in Explorer, that won't stop the user from copying files by other means, or by doing the "poor man's copy" by opening the document from the source location and doing a Save As to create a duplicate in the destination.

    If you want to prevent the user from copying files to a directory, use the NTFS security model. Withhold Create files permission in the folder, and users will be blocked from copying files into the directory in Explorer, Notepad, or any other program.

    Related: Shell policy is not the same as security.

Page 1 of 4 (35 items) 1234