August, 2009

If you're looking to get into some p/invoke action, you'd be well-served to check out the pinvoke wiki to see if somebody else has done it too. If what you need isn't there, you may end up forced to write your own, and here are some gotchas I've seen people run into:

• C++ bool and Win32 BOOLEAN are not the same as C# bool (aka System.Boolean). In Win32, BOOL is a 4-byte type, and BOOLEAN is a 1-byte type. [See also MadQ's remarks about VARIANT_BOOL.] Meanwhile, C++ bool is not standardized by Win32, so the size will vary based on your compiler, but most compilers use a 1-byte value. And then C# is even weirder: The bool is a 1-byte type, but it marshals as a 4-byte type by default.
• Win32 char is not the same as C# char (aka System.Char). In C#, char is a Unicode character (two bytes), whereas in C/C++ under Win32 it is an ANSI character (one byte).
• Win32 long is not the same as C# long (aka System.Int64). In C#, long is 64-bit value, whereas in C/C++ under Win32 it is a 32-bit value.
• If memory is allocated and freed across the interop boundary, make sure both sides are using the same allocator. It is my understanding that the CLR uses CoTaskMemAlloc/CoTaskMemFree by default. If your Win32 function doesn't use CoTaskMemAlloc, you'll have to teach the CLR which allocator you really want.
• When laying out structures, you have to watch out for alignment.

That last one is particularly gnarly on 64-bit systems, where alignment requirements are less forgiving than on x86. The structure declarations on pinvoke.net tend to ignore 64-bit issues. For example, the declaration of the INPUT structure (as of this writing—it's a wiki so it's probably changed by the time you read this) reads as follows:

[StructLayout(LayoutKind.Explicit)]struct INPUT {
  [FieldOffset(0)] int type;
  [FieldOffset(4)] MOUSEINPUT mi;
  [FieldOffset(4)] KEYBDINPUT ki;
  [FieldOffset(4)] HARDWAREINPUT hi;
}

This structure layout is correct for 32-bit Windows, but it's incorrect for 64-bit Windows.

Let's take a look at that MOUSEINPUT structure, for starters.

typedef struct tagMOUSEINPUT {
    LONG    dx;
    LONG    dy;
    DWORD   mouseData;
    DWORD   dwFlags;
    DWORD   time;
    ULONG_PTR dwExtraInfo;
} MOUSEINPUT, *PMOUSEINPUT, FAR* LPMOUSEINPUT;

In 64-bit Windows, the LONG and DWORD members are four bytes, but the dwExtraInfo is a ULONG_PTR, which is eight bytes on a 64-bit machine. Since Windows assumes /Zp8 packing, the dwExtraInfo must be aligned on an 8-byte boundary, which forces four bytes of padding to be inserted after the time to get the dwExtraInfo to align properly. And in order for all this to work, the MOUSEINPUT structure itself must be 8-byte aligned.

Now let's look at that INPUT structure again. Since the MOUSEINPUT comes after the type, there also needs to be padding between the type and the MOUSEINPUT to get the MOUSEINPUT back to an 8-byte boundary. In other words, the offset of mi in the INPUT structure is 8 on 64-bit Windows, not 4.

Here's how I would've written it:

// This generates the anonymous union
[StructLayout(LayoutKind.Explicit)] struct INPUT_UNION {
  [FieldOffset(0)] MOUSEINPUT mi;
  [FieldOffset(0)] KEYBDINPUT ki;
  [FieldOffset(0)] HARDWAREINPUT hi;
};

[StructLayout(LayoutKind.Sequential)] struct INPUT {
  int type;
  INPUT_UNION u;
}

I introduce a helper structure to represent the anonymous union that is the second half of the Win32 INPUT structure. By doing it this way, I let somebody else worry about the alignment, and it'll be correct for both 32-bit and 64-bit Windows.

static public void Main()
{
  Console.WriteLine(Marshal.OffsetOf(typeof(INPUT), "u"));
}

On a 32-bit system, this prints 4, and on a 64-bit system, it prints 8. The downside is that you have to type an extra u. when you access the mi, ki or hi members.

input i;
i.u.mi.dx = 0;

(I haven't checked what the PInvoke Interop Assistant comes up with for the INPUT structure.)

Since I'm obviously a glutton for punishment, I also helped read eighth grade essays on the same topic: Describe the qualities you consider to be those which make someone an adult. As always, remember that these are just the funny sentences/excerpts.

Let me tell you about my parents

• My dad looks older for one reason because he has facial hair.
• [My father] doesn't look young and muscular like most teens do.
• Some people are just plain dumb, but not my dad.
• My dad works at his extremely boring Job every day.
• If you slack on your work and lagger behind you won't have a good image when people think of you. Fortunately, my dad has covered this quality. Your dad is a slacker?
• He is a photographer so he has to work with a lot of people. Alot of those people are ignorant and stuffy politicians.
• My dad goes down to Seattle practically every day and does stuff I'm not totally sure about but he's a lawyer.
• My dad is an adult because [he] makes a budget so he can pay the bills on time, so we can keep our house and not get our car toed. It's called wheel alignment, and toeing is a good thing.
• My dad is an adult partly because he mad it through highschool and collage. I wonder whether he majored in anger management.
• My Dad... !! yayz!
• Three children and a wife can be a real hassle, some people can't even handle one.
• [Dad] recently had this mid life crisis I believe, he went kind of crazy and bought all this equipment for making music. He made his songs, and then kinda transitioned back to normal.
• He has engendered a family.
• In all te be considered an adult you have to be at or over 18. And I assure you my mom is well over 18.
• College also gave her [my mother] a social experience. This is important because it gave her a dose of adultery. Does your father know this?
• My mom is always my home skillet biscuit. Fo' shizzle.
• This means that my mom can drive and drink both things together only an adult can do. I take it your mom isn't a member of MADD.

Entering a no fun zone

• Being immature is like acting like a monkey or making fart noises with your mouth.
• [Adults should be] even keel (that means they don't go completely berserk if they don't get dessert)
• You can't slackoff like you did in middle school.
• No grownup is lazy.
• No more being babied by mommy.
• Movies: OK.
  Teaping: Not OK.
• When adults do work like working in the yard, they can go for hours on end!! Meanwhile we're in school going insane!
• Hitting isn't going to solve anything because it's just going to get you sewed and thrown into the big house. As ye sew so shall ye rip.
• Jobs can be stressful, boring, and horrible especially if you don't like your job.
• Her job brings in a luxurient amount of money.
• People who still live with their parents when they're 30 years old are not really adults.
• Go out get a job get a house and start a family. Don't wait or your gonna end up living with your parents!

It's harder than I thought

• Emotions make yourself weak and in life only the strong remain.
• I consider one person to be an adult. Wow, pretty exclusive club.
• To be a great adult you should try to resemble Jerry Seinfeld. Is that the one person?
• When I think of the word adult, I think of people like my Dad, my Mom, my teachers, and random parents I know. Like, a lottery?
• A adult should have a decent car like a shiney Corvette with shiney wheels.
• Being grown up is very rare and you might only see it a few times in your life. [...] Being [an adult] requires "the look." When you look into an adult's eyes and see wariness, regret, stern impassiveness, but most of all, intelligence.
• Believe in your yes and the Believing will make it happen, make you a good leader. Okay, you lost me at the "Believe in your yes" part.
• You need to be social to have friends and have fun as an adult. Instead of "cowering" in a dark corner of your high school reunion.
• You can dive to and from work.
• You have to pay the morgue.
• ... independent, responsible, and always goes beyond what he can do. He takes on more than he can handle?

Tautology corner

• An easy way to tell if someone's an adult is to see how mature they are.
• When you are a child, you have just about no maturity.
• There are numerous reasons why I believe the starting of college is the point you enter college.

Assorted commentary

• Finally, you have to be patient as an adult, because you might have kids you might not, but they are always everywhere.
• Parents are adults, clowns are adults, the ice cream man is an adult, most teachers are adults, and even teachers are adults.... Adults seem to be completely ubiquitous. About as ubiquitous as children, apparently.
• I think that every adult should have a job that gives descent money. How about moral decay money? Can I get that, too?
• When teenagers think "adult" most picture a grown up human being that can vote and drink alcohol until they puke. I'm having second thoughts on that moral decay money.
• Most adults fly off the handle at kids who are obnoxious or at their stupid boss at their stupid work. Having issues, are we?
• It would be hard to live in a world with immature adults. Yet somehow we manage.
• ... kids can get away with mistakes, because they're seen as little dogs who don't know any better.
• I'd like to live a low-stress life for a few more years before society demands my presence. Enjoy your doghood for a little while longer.
• If I am angry at someone or something, she is always by my side ready to fight. Is this an adult or an anime character?
• Without responsibilities, adults would be just like kids but much taller.
• Some people have all the experience in the world, but they have never analized it. Anal-ization sounds painful.
• Do you want to know what an adult is? Well, I hope so, 'Cause you're about to find out!
• Now we're getting off topic so let's wrap this up.
• There are probly more things to being an adult but what do I know I just trying to get passed 8th grade.

Misspelling corner. I've included more context; that may make the game a bit easier.

• She [Mom] also runs a house with no hunban on the weekdays.
• The thing I don't like is the chorse that she makes me do.
• ... doing buiesness 24/7.
• The Maturatity of Friendship
• ... a great person that reflex what a real adult should be like.
• You need responsibleaty to cope with the basicks of every day adulthood.
• There are also some fisual singhs of adults.
• Thoughs people have the maturity of an eighth grader.
• So the next generation has the change to become a secseder in life.

And just so you won't think all eighth graders are terrible writers:

• The truth of it all is your house is the central gravitational force, pulling bits of your life together.

"Why can't I pass a reference to a derived class to a function that takes a reference to a base class by reference?" That's a confusing question, but it's phrased that way because the simpler phrasing is wrong!

Ths misleading simplified phrasing of the question is "Why can't I pass a reference to a derived class to a function that takes a base class by reference?" And in fact the answer is "You can!"

class Base { }
class Derived : Base { }

class Program {
  static void f(Base b) { }

  public static void Main()
  {
      Derived d = new Derived();
      f(d);
  }
}

Our call to f passes a reference to the derived class to a function that takes a reference to the base class. This is perfectly fine.

When people ask this question, they are typically wondering about passing a reference to the base class by reference. There is a double indirection here. You are passing a reference to a variable, and the variable is a reference to the base class. And it is this double reference that causes the problem.

class Base { }
class Derived : Base { }

class Program {
  static void f(ref Base b) { }

  public static void Main()
  {
      Derived d = new Derived();
      f(ref d); // error
  }
}

Adding the ref keyword to the parameter results in a compiler error:

error CS1503: Argument '1': cannot convert from 'ref Derived' to 'ref Base'

The reason this is disallowed is that it would allow you to violate the type system. Consider:

  static void f(ref Base b) { b = new Base(); }

Now things get interesting. Your call to f(ref d) passes a reference to a Derived by reference. When the f function modifies its formal parameter b, it's actually modifying your variable d. What's worse, it's putting a Base in it! When f returns, your variable d, which is declared as being a reference to a Derived is actually a reference to the base class Base.

At this point everything falls apart. Your program calls some method like d.OnlyInDerived(), and the CLR ends up executing a method on an object that doesn't even support that method.

You actually knew this; you just didn't know it. Let's start from the easier cases and work up. First, passing a reference into a function:

void f(SomeClass s);

...
   T t = new T();
   f(t);

The function f expects to receive a reference to a SomeClass, but you're passing a reference to a T. When is this legal?

"Duh. T must be SomeClass or a class derived from SomeClass."

What's good for the goose is good for the gander. When you pass a parameter as ref, it not only goes into the method, but it also comes out. (Not strictly true but close enough.) You can think of it as a bidirectional parameter to the function call. Therefore, the rule "If a function expects a reference to a class, you must provide a reference to that class or a derived class" applies in both directions. When the parameter goes in, you must provide a reference to that class or a derived class. And when the parameter comes out, it also must be a reference to that class or a derived class (because the function is "passing the parameter" back to you, the caller).

But the only time that S can be T or a subclass, while simultaneously having T be S or a subclass is when S and T are the same thing. This is just the law of antisymmetry for partially-ordered sets: "if a ≤ b and b ≤ a, then a = b."

I didn't participate in the reading of the seventh grade essays, but I did get some of the more entertaining sentences from that batch. As you may recall, the topic was to describe the qualities you consider to be those which make someone an adult. Students were given 90 minutes, plus one additional hour upon request, equipped only with paper and pencil.

Remember, these are just the funny sentences/excerpts. Do not assume that all students write like this.

Better get your butt in gear

• Grown ups should have accomplished something great in order to be considered an adult.
• Also, all adults in some small way contribute to the global economy.
• One of the creepy qualities of adults is that they have an answer for everything.
• There should be a test or classified training facility or something to prove you're adult.
• They could make a segment that tests your adultivity on the SATs.
• Adults should not get drunk or drink juice or pop.
• You can't get all fat, you have to be resposible and exercise enough so you stay fit and inshape.
• An adult never wines crys cheats and always is kind.

Let me tell you about my parents

• My dad is so responsible, I think because he has so much responsibility he has to be responsible.
• I almost feel as if my heart and my dad's heart beat as one.
• If we're doing something dangerous and "stupid," he will try and I repeat try to stop us.
• A way he is healthy is that I know he brushes his teeth and does not smell like a pig.
• My mom is an adult. She's tall, a bit old, and fully grown.
• Parents probably seem the most like adults because they have been around longer than you, but not long enough to make them senior citizens.
• My mom is so responsible she will fix mistakes someone else did. NOW.

How'd they get that way?

• Adults, What Are They?
• Another way to tell that someone is an adult is that someone could be very old like if they sleep too much or if their hair was white and if they barely have any.
• Adults are not only smarter but they are very boring too. They are always acting so adult-like in a way that is so boring to me, but apparently amusing to other adults like them.
• College is where they learn to swear and smoke and drink. As adults I hope you will all outgrow all that trash.
• It somehow occurred to me that once you hit a certain age you just stop getting embarrased.
• An adult is mysterious or maybe even secret. You can't tell what they are thinking in that developed brain of knowledge and thought.
• Some adults are just so boring that you wonder if they can actually laugh.
• Soon they are throwing their thoughts into the world like cerebral dynamite.

Assorted commentary

• When I think of an adult, I think of really tall people who are married.
• It's not my fault we age so quickly.
• If you are not an adult at 43 then I don't know what you are.
• Being an adult can be a very stressful way of living.
• The person I see as being adult is M.C. Hammer.
• My cousin respects me until his friends come over.

Misspelling corner. I've included more context; that may make the game a bit easier.

• Adults have excellent manors.
• If you aren't responcable, then certin scenrios might happen.
• Adults are often considered excepted members of society.
• He doesn't guess at things, he does the grounwork to get a more accurate answer or sulusion.
• Shes coquered life shes had frivolious times and malignant dilemmas.
• Just play it by year.
• Teachers are dedinately very pioused adults.
• I think a little bit of dissiplin isn't bad for the soul.
• ... and when I am rushing down the court trite and sweaty...
• He is also a over a cheaver.
• You have to be a rolmatal for others.
• That makes you look irrespunkabl.

And just to show that seventh graders are also capable of good writing:

• Most people think that right when you turn 18 you are an adult but truly it's one small step in a long hard journey.

A few years ago, Abhinaba wondered why FlagsAttribute didn't also alter the way enumeration values are auto-assigned.

Because attributes don't change the language. They are instructions to the runtime environment or (in rarer cases) to the compiler. An attribute can instruct the runtime environment to treat the function or class in a particular way. For example, you can use an attribute to tell the runtime environment that you want the program entry point to run in a single-threaded apartment, to tell the runtime environment how to look up your p/invoke function, or to tell the compiler to suppress a particular class of warnings.

But changing how values for enumerations are assigned, well that actually changes the language. An attribute can't change the operator precedence tables. An attribute can't change the way overloaded functions are resolved. An attribute can't change the statement block tokens from curly braces to square braces. An attribute can't change the IL that gets generated. The code still compiles to the same IL; the attribute just controls the execution environment, such as how the JIT compiler chooses to lay out a structure in memory.

Attribute or not, enumerations follow the same rule for automatic assignment: An enumeration symbol receives the value one greater than the previous enumeration symbol.

Welcome to CLR Week 2009. As always, we start with a warm-up.

The String.Format method doesn't throw a FormatException if you pass too many parameters, but it does if you pass too few. Why the asymmetry?

Well, this is the type of asymmetry you see in the world a lot. You need a ticket for each person that attends a concert. If you have too few tickets, they won't let you in. If you have too many, well, that's a bit wasteful, but you can still get in; the extras are ignored. If you create an array with 10 elements and use only the first five, nobody is going to raise an ArrayBiggerThanNecessary exception. Similarly, the String.Format message doesn't mind if you pass too many parameters; it just ignores the extras. There's nothing harmful about it, just a bit wasteful.

Besides, you probably don't want this to be an error:

if (verbose) {
  format = "{0} is not {1} (because of {2})";
} else {
  format = "{0} not {1}";
}
String.Format(format, "Zero", "One", "Two");

Think of the format string as a SELECT clause from the dataset provided by the remaining parameters. If your table has fields ID and NAME and you select just the ID, there's nothing wrong with that. But if you ask for DATE, then you have an error.

I was out of town for the grading of the seventh grade essays, so I pitched in with the sixth grade essays instead. The students were asked to think of an adult and describe the qualities that make that person an adult. This topic was not very well received by the students, who deemed it uncreative and boring. While I understand their lack of enthusiasm, it's also true that for most of your life, you're going to have to write on topics that are uncreative and boring (and the stakes are going to be higher), so you'd better get good at it.

The difference in writing skill between sixth and seventh graders (between eleven year olds and twelve year olds) is quite noticeable. Many of sixth graders could not get past the literal definition of the word adult, describing the qualities that make an adult purely in terms of biology: Age, height, strength, puberty, armpit hair. Many others focused on accomplishments or privileges that distinguish adults from children: Advanced education, having a job, knowing how to drive a car, and being able to stay up late without getting yelled at.

Remember, these are just the funny sentences/excerpts. Do not assume that all students write like this. The assignment is given under standardized test conditions: 90 minutes with nothing but pencil and paper, with one additional hour available upon request.

The easy life

• After collage, you could just go home and relax for the rest of your life. And you wonder why scrapbooking is so popular? It's because everybody goes to collage!
• Adults go on vacation to international countries and play golf. Gosh, I wonder if this student comes from a wealthy family.
• My dad is a big fan of football. Who's not?

Check your fun at the door

• Adults don't like to do anything fun 60% of the time. I think I'm getting shortchanged on the other 40%, too.
• They talk and talk and talk, that's all they do.
• When an adult takes you somewhere it is usually to a depactment store.
• My mom was so busy she had to step up her gear to get it all done.
• My mom is nice, she likes to get new kitchen supplies and carpet.
• Just like kids, adults still make mistakes and just want to have fun.
• Being an adult means living above the influence.
• Adults don't do stupid things like throwing wild house parties 24/7.
• Mature people order off the adult menu at Red Robin and do not order jumbo sundaes with extra cherries.

Responsible behavior

• My mom is responsible because she cleans the house before anybody tells her to.
• My dad is good sport. If he wins something, he doesn't say, "nanny nanny boo boo."
• Being mature is one big part about being an adult because if no one was mature then we would be at war all the time. Instead, adults just talk and talk and talk.
• My mom cleans the house because my dad doesn't. Are you suggesting that your dad isn't an adult?
• Being a civilized adult is simple, and all adults have it just not all the time.

Let me tell you about my parents

• She's now a mail women.
• My dad is an adult because of his hair. He is losing his hair, and I hope he will not be bald soon.
• He has a wife (my mom). Thanks for clearing that up.
• My dad is really smart. He's been married three times. Most people are satisfied to be only one third as smart as him.
• A good dad makes his own meals, and who doesn't like a guy that knows a thing or two about the stove?
• My dad is tough. He is not easily scared by spiders, lightening, or the dark. I'm assuming that lightening was a spelling error, but who knows?
• My mom helps me appreciate that I don't live in a third world country. For example, she frequently reminds me that there are starving children in Africa.
• I hope one day my mom will live longer than ever before. "Hey Mom, have you ever been this old before?" "How about now?"
• My mom is the first adult I ever met.
• My parents are not overly protective. They let me eat raw cookie dough.
• When I saw my Uncle Mike in Texas, I knew he was an adult.
• Sarah has many personalities that make her an adult.

Assorted commentary

• Adults can't whine unless their car gets totaled, or their house burns down.
• But I think of you more as an adult when you have to shave your back hair.
• My dad is one of the most manly adults I know. Must be the back hair.
• Eight-teen is a huge age!
• OK, you've survived the first paragraph. Good, because here comes the second one.
• Most adults I know are pretty smart. Not Albert Einstein smart, but common sense smart.
• Adults are great people unless they aren't what they shouldn't be.
• Every single adult has gone through puberty or at least half way. I believe the ones that went through only halfway are known as 'Frat boys'.
• Best opening sentence: When you grow up to be an adult, you get armpit hair.

Concluding thoughts

• I also think I'm going to have a hard time when I be an adult.
• And those are just some snidbits from my brain.

Misspelling corner. I've included more context; that may make the game a bit easier.

• He doesn't goof around when he is so post to be doing work.
• Responsibility is when you have to make the right desigin.
• Being grown up is never goffing up.
• All the school gets the money from us for fiead hips and fun razors and the suplise. I'm trying to imagine what a fun razor is.
• Less mature human beans are not very keen and when doing extensive work they throw tantrums.
• They should be aloud to drive. Speak up! I can't hear you driving.
• When you become an adult you get fatiol hair.
• She doesn't wine like a baby. Hey, baby, how about another glass of chardonnay?
• My dad also plays motable instruments.

Other remarks on student writing:

• Most essays followed the standard introduction formula "There are three qualities that make a person an adult. Those qualities are A, B, and C." It's a treat to find an essay that opens more creatively, but alas, the essays with excellent introductions failed to maintain the quality level for the rest of the essay.
• You can't just write "And that's why X" to conclude your paragraph if you never actually explained why X. I read many paragraphs that took the form "Adults have quality Q. Quality Q means that XYZ. That's why adults should have quality Q." The paragraph did nothing to explain why quality Q is an important one for adults to have; it merely stated and defined it. One teacher explained to me that this isn't a conscious writing choice but is rather simply a bad habit. "I've written a bunch on topic X. Now I need to wrap it up. And the way you wrap it up is to write, 'And that's why X.'"

And that's why I read student essays.

The other night, I was playing a friendly game of Scrabble®, and I managed to play BEANIER* (meaning "with a stronger flavor of beans") onto a triple-word score, crossing the B with an open Y, scoring over 100 points in the process. This sufficiently demoralized the other players that the game turned into "play anything that vaguely resembles a word, with creative spelling encouraged."

It turns out that BEANIER* is not listed in the online versions of the SOWPODS or TWL Scrabble word lists, although I made the move in good faith. If the others had thought to challenge, they would've succeeded.

My brother and I play Scrabble with very different styles. I'm not so much concerned with scoring (although I certainly try to make high-scoring moves) as I am with having a pretty board with a lot of intersections and clever words. I treat Scrabble as a collaborative effort that happens to have a winner at the end, in the same spirit as shows like My Music or Says You. As a result, I don't pay too much attention to whether I'm opening easy access to a triple-word square, and I will forego a higher-scoring play in favor of one that uses a funny word or which connects two parts of the board. If you look at my scoresheet at the end of the game, it consists of a lot of medium-scoring moves (and a few really pathetic ones), with maybe one "super-move" per game where I play a bingo or otherwise manage to rack up a lot of points at one go.

My brother's approach is much more methodical. He doesn't play a very flashy game; he just focuses on scoring twenty or more points per move. If you look at his scoresheet, it's just a slow, steady climb to the final tally.

This means that when we play, it's a competition between the tortoise and the hare. (I'm the hare.) Will my "super-move" be enough to hold off the steady erosion of my lead from the constant barrage of strong moves? Usually, the answer is No. Slow and steady wins the race. But I like to think I have more fun.

One of the flags you can pass to the IShellFolder::CompareIDs method is SHCIDS_CANONICALONLY. This flag means that the method should determine whether the two pointers refer to the same underlying object, and if they do not, then it should determine which one should come first by whatever mechanism it wants. It doesn't matter which one is declared as coming before the other one, as long as it is consistent.

I like to think of this as the moral equivalent of the Unicode ordinal comparison. In both cases, you use the comparison if you have two items that you wish to keep in sorted order, but you don't care what the ordering rules are, as long as they are consistent. In fact, all you care about is consistency, and you're perfectly happy to sacrifice readability for speed. The resulting sorted list won't be displayed to the user; all you're going to use it for is locating the item later.

You can think of this as the moral equivalent of the NTFS file name sorting algorithm. In both cases, the items are sorted not so that the user can find them, but so that the program can find them.

Back in the days before perl ruled the earth, regular expressions were one of those weird niche features, one of those things that everybody reimplements when they need it. If you look at the old unix tools, you'll see that even then, there were three different regular expression engines with different syntax. You had grep, egrep, and vi. Probably more.

The grep regular expression language supported character classes, the dot wildcard, the asterisk operator, the start and end anchors, and grouping. No plus operator, no question mark, no alternation, no repetition counts. The egrep program added support for plus, question mark, and alternation. Meanwhile, somebody went back and added repetition counts to grep but didn't add them to vi; somebody else added the \< and \> metacharacters to vi but didn't add them to sed. POSIX added repetition counts to awk but changed the notation from \{n,m\} to {n,m}. And so on.

No two programs use the same regular expression language, but they overlap sufficiently that you can often get by with the common subset and not have to worry about which particular flavor you're up against.

Until you wander into the places where they differ.

From: John Jones
Subject: Problem with regular expression

I'm trying to write a regular expression to match blah blah blah.

From: Jane Smith
Subject: RE: Problem with regular expression

I think this will match what you want: ^Z@1&*B*!34

I just ran my hand randomly over the keyboard to generate that fake regular expression. The scary thing is, at first glance, it is not obviously not a regular expression!

From: Chris Brown
Subject: RE: Problem with regular expression

Try $)(#$C)*#

From: John Smith
Subject: RE: Problem with regular expression

Thanks, everybody, for your suggestions, but I can't get any of them to work. For example, I can't get any of them to match against this string: blah blah blah blah.

At this point, people chimed in with other suggestions, confirming that John doubled the backslashes, that sort of thing. John posted his test program, and then the reason was obvious.

From: Jane Smith
Subject: RE: Problem with regular expression

Oh, you're using CAtlRegExp. In that class, \w doesn't match a single character; it matches an entire word. You want to use \a instead.