annotated in parsing

My Dev team is bigger than yours.

santoshz — Thu, 20 Oct 2005 17:41:00 GMT

On my drive in today I heard a piece on NPR entitled "Corporate Influence Seen Harming Entertainment Industry"

Some analysts and producers say the entertainment industry is struggling because of the corporatization of movie and TV studios. The pressure to keep stock prices and revenues up contributes to less risk-taking on the screen.

And I kept thinking how true much of thepiece was about software too. Which then reminded me of Paul Graham's essay "Made in USA"

[Here is a couple of snippets, but I am so tempted to paste almost all of it here]

... It sounds like making movies works a lot like making software. Every movie is a Frankenstein, full of imperfections and usually quite different from what was originally envisioned. But interesting, and finished fairly quickly.....

... But it's not just that software and movies are malleable mediums. In those businesses, the designers (though they're not generally called that) have more power. Software companies, at least successful ones, tend to be run by programmers. And in the film industry, though producers may second-guess directors, the director controls most of what appears on the screen. And so American software and movies, and Japanese cars, all have this in common: the people in charge care about design-- the former because the designers are in charge, and the latter because the whole culture cares about design...

In the NPR piece, Kim Masters compared movies with television and talked about how the issue of corporate influence is quite different for the two. Since I work for a pretty big software company, I got to wondering if we make movies or television series.

With movies you tell one good story in one sitting (sometimes two, but rarely in a triology). On the other hand with a TV series, you take a context and then stretch out a series of stories that keeps the viewer coming back for more goodness.

If, like my company does, you version your software, then I am inclined to believe I in the TV series business, not the movie one.

So my question is, what is the best size for a software product team to produce a great series? How big does your corporate parent have to be before it starts getting in the way between you and your loyal viewer?

Old school problem solvers wanted

santoshz — Fri, 04 Mar 2005 17:25:00 GMT

My commute was longer than usual today, so I had some extra time to read and think.

The April 2005 issue of Software Development (available only in print form at this moment – See! advertisers still prefer eyeballs on paper to eyeballs on pixels) has an interesting piece of “Feedback” (why not just call it Letters to the Editor?) from a James S. Taylor.

In that, James’ son says “Fewer and fewer projects need problem solvers of the old school. What’s needed are a crew of programmers who can implement the deliverables in the time available, and if it takes gigahertz or terabytes, so what? They’re cheap. Only when you’re deep in some embedded system where you can’t solve the problem by throwing hardware at it do you require a real problem solver, and no one does that any more.”

I liked James’ short letter because it hit two points I have been thinking about quite a bit lately, and it did so in a thought-provoking way. The first, as alluded to above, is about “What type of engineering is Software Development really?” The second, is about “What makes a good hire?”

Anyway, I was reminded of a comment that I want to share here. It was made by a fellow-employee when we took our Interview Training. (Yes, we do take interview training at Microsoft. And I personally think that it a good thing, because of how important the whole hiring process is. If you rank the things you do in terms of their impact on your company per minute you spend on it, there is nothing that comes out higher than interviews.) After two or three hours of being drilled on the fine art of judging a candidate, this person had had enough and threw up his hands. He said "If you come to me and tell me that you are one heck of a runner, do I ask you ‘Tell me about a time in your past when you ran well?’ or ‘If you were going to have to run this race in five minutes, and you could not find your track shoes, how would you do it?’ or ‘Tell me how you would deal with such and such on a team of runners?’ Dammit, NO! I'd say ‘Okay, let’s see you run. Then I will know if you are a good runner or not.’ "

And something that James Taylor said, suddenly went click, and I realized that just like an actor, or singer, or other performer has to show up at an audition, get in a long line, and then strut his or her stuff for a few minutes, there comes a point during every Microsoft interview (the dreaded whiteboard) when you have to strut your stuff. And each of the interviewers sitting there watching you has to decide inidividually whether you demonstrate creativity and intuition, and can leverage it effectively to solve problems. (That was James' point, that these are what predict a good programmer hire.)

If you want to strut your stuff, we still have some positions available in C# (both in Test & Dev, junior and senior). Unlike the stage audition, I believe we also give you bus-fare home, so what's to lose?

Cool!

santoshz — Wed, 09 Feb 2005 15:36:00 GMT

Yesterday, I was so pleasantly surprised by a product, that I actually went and got all my colleagues, and showed this to them.

I was installing Virtual PC for the first time, and when it came time to configure my first virtual machine, a dialog asked me to name it. So, quite predictably, I typed in "MyWin98".

(Yes, we still do support the C# compiler on Win98, and part of my team’s job is to make sure all the newest and fanciest and greatest continues to work on your ancient heap. In this case, I needed to install Win98, Second Edition, Japanese. When I said ‘your’ back there, I was including myself since I still have a machine at home running Win95 that gets daily use. But I digress…)

The next step of the dialog was to select an Operating System for the virtual machine. Here comes the cool part.

Windows 98, had already been selected in the drop down list!

I have never seen a Microsoft product anticipate me this well. Of course, my office-mate’s (soon to be my ex-office mate because he gets his own office next week) first reaction was to try a host of other names to see if he could trip it up. Typical QA….. doesn’t want to know what it can do, just wants to find out what it can’t. Well, suffice to say that I was happy it recognized versions of Win2k3 too.

Thanks Ben, and/or those on the Virtual PC team who thought this up and took the trouble to implement it. (I can picture some Tester in a design meeting rolling her eyes and thinking, "Yeah, now we have to figure out what Norwegians like to call their personal electronic friends and test all variations of that.") Actually, I am willing to wager that a tester thought up this idea."

Wanna have fun?

santoshz — Sat, 05 Feb 2005 09:13:00 GMT

If you know what having fun means for you, tell us at http://msdn.microsoft.com/vcsharp/jobs

(Tell them the guy on the park bench sent you. And if you are tempted to send Gretchen flowers, ... I'd wait another week or so, and do avoid daffodils, especially the sidewalk variety)

It’s not sexy. But it sure is fun.

santoshz — Fri, 04 Feb 2005 05:56:00 GMT

Don’t tell anyone, but it is spring here in Seattle. The daffodils in the patch of dirt outside my front door have started peeping out. A blossom tree on my way to work is in full bloom. And I saw an e-mail thread yesterday trying to get a crowd together to go to the nearest ski-able snow - in Tahoe. All this feels strange, because all the Valentine’s Day books I have been reading with my son recently have had pictures of snow in them.

Maybe it is only adults who see these incongruities. After all, during my childhood in the tropics I read my fair share of American and British children’s books (I won’t date myself by telling you which ones), but I don’t recall ever being perturbed that I was not likely to ever see any real snow in my life.

Segue to the other big sign of new beginnings in the Puget Sound. At work, we recently concluded our first Test Day – an in-house, one-day conference devoted to Testing. It was like any regular conference, except you did not have to catch a plane and stay in a hotel. It even had a little trade show. However, it was different from a regular conference (like the one I attended in Baltimore back in Dec.) in that there were some very good career track sessions where senior Testers talked about career paths and options. Another difference was that you did not get any time to stand around and chat with people who are total strangers, yet with whom you share this special bond. (Like being on the slopes or the greens, I guess.)

Of late, I’ve also noticed a lot more blog activity from the Testing community here are Microsoft. (Of course, maybe it was always there, but, as usual, I wake up to the party when it is just finishing.) (No, I am not going to waste any more time searching for a bunch of links) A lot of this blogging seems to be about what it means to be a Tester at Microsoft. It is almost as if this community has hit adolescence – the outward swagger along with the internal nerve-wracking quest for self-identity.

This raising of the public profile of Testing is interesting, because I normally think of Testers as being on the dour end of the spectrum. Of course, I work daily with all sorts of people - the chaotic, the orderly, the quiet, the flamboyant, those who rush into action, and those who plan excessively – so I should not be tarring them all with the same brush. But as a breed I think we Testers are conservative, and a bit pessimistic. We are not the creative energy of an organization. Rather we have the reputation of being the naysayers. We don’t build anything, yet constantly argue that it is broke. It is our business to know what can go wrong (and like Tom Ridge we even color code the status of our features). We are not people who can do a Steve (Ballmer or Jobs, your pick) in front of the faithful.

The spurt of blogging is positively uplifting when compared with the reading I have been doing recently about Software metrics, and Defect Analysis: 6σ, CMM, Blackbelts, TSP, PSP, ODC, QGM, TQM, and on and on and on. So much process! Of course, if you are Boeing (you forget, I live in Jet City), or GE (the six sigma kings who make the engines for Boeing) all this matters a great deal. It is easy for you and me to see what an airplane falling out of the sky can do to their bottom lines. (Mind you, with each passing Windows security bug, Microsoft moves one step closer to that. And if Air New Zealand takes delivery of the first 787 - yes, the 7E7 got its official name last Friday - before we ship Longhorn, then maybe we should consider changing things around so we become more like Boeing.)

All this - the Baltimore conference, Test Day, Testing blogs, Quality Processes, Boeing & GE – all this finally kinda gelled in my mind this morning as I drove in to work, We – the Testing community at Microsoft - are different. We do live in a different land – where the trees are in bloom and there is no snow in February. We do feel out of place amidst the regular Testing literature and conferences. Maybe we just need to be children again, and not be so bothered by the difference.

Here’s why we are different. In my mind, software projects at Microsoft are bit chaotic. (you may detect the lingering after-taste of Paul Graham’s Made in USA essay here. I myself did not realize it until many hours after I first wrote this para.) The product grows bit by bit around bursts of intense creativity by individuals or very small teams. Designs change for technical reasons. Market pressures come to bear. Stuff gets ripped out. New stuff gets put in. Things that were previously not possible, suddenly become. Stuff that was carefully planned gets canned. The mass of features, both old and new, aggregate in to the final package, with interconnections to the rest of our product line. Almost from the first sketch on a napkin, to the final shrink wrap on the glitzy box, we in Test are there -hovering over every process, like clucking mother hens worried about what almost surely is going wrong right now. We are constantly trying every thing we can to make sure that there is nothing wrong in the product, as well as everything right in it.

If you think of a software project as akin to building a large, new public garden, then we are the groundsmen in the ever-shifting landscape. If we don’t choose the seeds, and we don’t plant or water them, and we don’t lay down the pathways through which the users will enjoy our garden – since there are other people (maybe even non-dour ones) who do this, and who do it extremely well – then what is it we do then?

Well, when we first see the plans we might remark that service trucks might not be able to get to the back of the buildings. We might observe that the sidewalk is not wide enough for the crowd on a busy day. We might point out that it is not intuitive as you round the first corner that you should keep going to the right, rather than cross over to the left. Maybe that tree is going to leave a lot of leaves on the ground in the fall. And if you dropped in a paved path just here, it would make testing much quicker easier.

As construction gets under way, we are there every day, sitting in the benches, walking down the pathways, reading the signs. And each time we hope we see it just the way the first-time visitor would, and just as the old-timer would too. Where is the nearest rest-room? Is it sign-posted? What if you came at the sign from the other side? What if you did not read English. What if you could not see? Why do I always have to go all the way around to get to it? Can a wheelchair go on this path? Does that tree block the view of the lake from this bench? Is the bench back in the same place after they ripped it out to lay the new plumbing? Did they put the sign back too? Did they paint it just the right color? What would happen to the lake if it rained four days straight. What if it rained four days straight with one foot of snow on the ground? What if the rain was warm? What if the power went during the snow and did not come back when it started to rain? What if someone held a banquet at night? Or a rock concert?

When the first alpha and beta visitors come in, we wait with bated breath to see what they will find. Oops, no one caught the possibility of splinters on those benches. Hey, we used that same wood for the picnic table across there. The signage is not working, we are going to have to re-think this. Note to self: Sign got repainted and repositioned - Did they stick with the correct color? Is the post blocking any views in the new place? Damn, they extended the parking lot without extending the sidewalk; too late to fix that now. And so on and on it goes, with more and more tension until the last service truck (and Program Manager, and Developer, and Architect, and Usability Expert and User Educator, and Setup Team member) leaves before opening day. Then, it is just us and the final product, and a few short hours for the final walk-through. It is almost like the mother of the bride, proud of what waits on the other side, yet reluctant to let her baby go. (For a blow-by-blow account of the last week of our first VS 2005 Beta, see this blog by Soma, ex-tester, now VP of our Division.)

That’s what we do – except we ‘walk’ and ‘look’ and ‘read’ and ‘sit’ and ‘cause storms’ in a real software application, using little code golf carts we write, and little code binoculars we write, and little code rainmakers we write, and big harnesses we write that let us do this again and again for months without going crazy.

But if you’ve ever worked in QA - as is very likely if you are this far into my ramble - then all this is old hat for you. “Tell me something new” you say.

The key is - and this came to me only today – the key is that while we are inextricably linked into each part of the process, we Testers at Microsoft also need to be as transparent and frictionless as possible. We can’t weigh down the creative, and at times chaotic, flow with layers and layers of process that bring the whole creaking machine to a halt.

It is our role to apply only just as much pressure as necessary at each step to ensure a high quality product, on time and on budget. Some times this could be the barest of nudges. Sometimes we might have to refuse to sign-off. When we do the former, we our doing our job well – we’ve been looking in the right places, at right time, in the right way, and we know exactly where, when and how hard to nudge. If we find ourselves doing the latter, we obviously failed to intervene adequately at all the previous points.

So, … do we know how to do this process-free, just-in-time QA? Heck, no! You never know how to do such things. You just constantly, constantly, constantly try to get better, and then one day it looks to others like you know what you are doing. It’s then you realize that you are no longer an adolescent – You know what music you like, what clothes you like, what food you like, and what having fun means – especially while at work.

Of course, since this is Microsoft and the landscape is an application, you have to understand software. Nay, it has to be in your blood and at your every synapse. You have to be able to design and write code like the best. Otherwise you would not know where to look, how to look, and when to look for bugs. You would not be able to get it all done in time, with confidence. But in Test it is a lot, lot more than that. It’s also about people, about processes, about organizations, about customers. It’s both about keeping bugs out of the product and about getting those that are in fixed.

It’s about being transparent. It’s about leaving your mark, with not a line of shipping code to your name.

It’s not sexy. But it sure is fun.

[I owe this blog to Suma on the C# IDE team. This piece began as a conversation with her, and she urged me to blog it, and then stuck around to provide very useful comments on a draft version.]

Guilty!

santoshz — Thu, 03 Feb 2005 06:43:00 GMT

The defense would like to enter a plea of guilty to the charge of “1^st degree hypocrisy.”

Since my last posting, I have put on weight, eaten more chocolate than I care to confess, and had not an iota of exercise (what is the official unit of exercise? You can try here, but I did not have much luck.) Meanwhile the official recommendation for exercise has gone up again, to 60 minutes a day. To which my office-mate who runs daily, come pain or rain, remarked “Well, now the average American gets 60 minutes less exercise than they ought to.”

Today’s scary news: Would $12000 of after-insurance medical expenses be enough to send you into chapter 7 or 13? (Why call it insurance, if it is does not provide catastrophic coverage?)

(Btw, Search - don't you like it how old words take on new meanings - has a long, long way to go. Five sentences into this piece, and I have already spent at least 30 minutes in search-engine land with nothing to show for it. I have one question (unit of exercise) still unanswered, and two (Federal exercise recommendations and the Harvard Consumer Bankruptcy Project) for which I cannot track down the original sources. As for a link to a recent article about Google versus MSN, I guess maybe there just aren’t any. J)

I was going to write about Testing at Microsoft, but now that’s going to have to wait for another day (Thanks MSN, thanks Google).

Obesity inversely linked to income

santoshz — Tue, 17 Aug 2004 17:10:00 GMT

It's been pretty obvious to my wife and myself that to eat 'well' in the US you need to be 'rich.'

Finally, a news story from NPR's Morning Edition that quantifies this succinctly. (Obesity Often Linked to Income Americans eat about 50 percent of their meals in restaurants and fast-food counters, a habit tied to the nation's obesity epidemic.)

The median household income in the US is currently hovering at about $42,000. If your family earns more than this, you could consider yourself rich - certainly not poor.

The median family apparently spends about $25 per person per week on food (($3.50/day or $1300/year - approximately 12% of gross income; probably closer to 25% once you have deducted taxes, insurance premiums, and housing). Thus half of the United States can afford about $1.80 per 1000 calories per day, assuming a 2000 Calorie diet.

Now come the interesting numbers.

If you limit yourself to 2000 Calories a day and to Oreo cookies (not that you would!), you can manage on $1.50. If you stuck to apples, that would cost you about $8-$10 a day for the same 2000 Calories.

More realistically, bread, chips, cookies, and off-the-shelf prepared foods run about $1 per 1000 Calories. Whereas fresh fruit, fruit juices, and fresh produce will cost you about $5-$10/1000 Calories. If you add lean meats and fish to the mix, your costs go even higher. The Atkins diet has apparently been costed at $15 per person per day (at $5,500 per person per year, that's more than four times what the median American can afford!)

This more or less confirms another new story I remember from many years ago, where one of the people interviewed ate 3 meals a day, 7 days a week, at McDonalds, because that was all he could afford.

If you assume that a good, employer-provided health insurance plan costs about $3,600 today, that is 3(!) times what the median American spends on food. (Admitedly, the health plan usually covers the whole family - still the cost of food and health care are now almost comparable.) For each overweight/obese person in the family, add a 10%-20% premium for additional healthcare expenses.

Now that you can get your HMO to pay for your quit-smoking and weight-loss programs, how long before they start contributing to your groceries too?

Follow this link for more news stories.

New car models and not lost in translation.

santoshz — Mon, 28 Jun 2004 22:23:00 GMT

Two compare-and-contrast scenarios consumed some idle cycles in my brain over the weekend.

One was a recurring “What if?“ I tend to ponder over. What if someone with a product cycle or two from the automotive industry under his/her belt moved to software development? What new ideas would they bring? (Obviously I see a good number of parallels between the automotive and software industries.)

The latest bout of this train of thought was provoked when I took our Subaru Legacy wagon in for an oil change, and admired the new 2005 model. Little did I know that they were offering me $25 to take a test drive - only later that day did I happen to find the coupon in the latest installment of Subaru's little monthly mailing to owners called Drive - they devoted the entire issue to a summary of the 5-year process that brings a new model to market. Of course it was all PR and marketing, but it did get me thinking again about the parallels, especially since the product I work on is coming to the tail end of a similar process as it readies itself for the release of its 2005 model.

[Place holder for my thoughts - if I ever get the time.]

The other train of thought was prompted by recent discussions with Daigo and some basic reading on compilers that I am currently doing. Daigo is very particular that code read well - and he is very clear when he reads code if is well-written or not. This is an aesthetic I would dearly love to have (your suggestions on how to develop this are most welcome). I do have it for English - both fiction and non-fiction (at least I know, like Daigo, what I like and don't like). Now English is a dynamic language, with a vocabulary, a grammar, and an idiom (relevant to your locale) - If one were to cultivate a style that was free of ambiguity, easily revised, edited, and translated, what would that be?

[Yet another place holder .....]

If it aint broke.....

santoshz — Wed, 16 Jun 2004 14:34:00 GMT

At what point does a system become so broke that you can't fix it, and must build a completely new one from scratch?

As one working in the trenches of QA/Test, I lean towards the viewpoint that we do what we can to uncover defects and quantify the level of quality, but by ourselves we can't ensure the customer experience. Hence my sympathy last week with Mike Unum. Of course, we constantly try to be more effective, efficient and innovative - but always within an existing process.

However, if, as may well be the case, the proportional cost of QA/Test in the product lifetime is going up, then do we still have time to reverse the trend before it comes time to sell our processes to management trainers to use as case studies? Or should we be reading the writing on the wall ...? (I acknowledge that software quality keeps getting better. But is the economic cost sustainable?)

Why the sudden pessimism?

Driving in to work today an item from the new headlines got me thinking - the quality of health care is steadily getting better for those whose companies can afford the double-digit inflation. And those who cant?

The Census Bureau estimates that over 43 million Americans did not have any health insurance in all of 2002. Families USA, looking only at the Census Bureau data, has concluded that almost 82 million Americans under 65 (1 in 3) did not have health insurance at some point in 2002 and 2003 - and a good number of these were employed, often with good paying jobs. (And this does not even address those who have insurance, with reduced coverages.)

Among the key findings of their report:

One out of three people in the United States under the age of 65 went without health insurance for all or part of the two-year period from 2002-2003 (approximately 81.8 million uninsured people—32.2 percent of those under the age of 65).
Two-thirds (65.3 percent) of the 81.8 million uninsured people were without health insurance coverage for six months or longer during 2002-2003. Over half (50.6 percent) of the uninsured were without health coverage for nine or more months.
More than four in five individuals (84.5 percent) who went without health insurance during 2002-2003 were connected to the workforce in December 2003—78.8 percent were employed, and 5.7 percent were actively looking for employment.
A quarter (25.2 percent) of working individuals and their families with incomes between 300 and 400 percent of the federal poverty level (from $55,980 to $74,040 a year for a family of four in 2003) were uninsured.
Hispanic and African American people were much more likely to be uninsured compared to white, non-Hispanic people: 59.5 percent of Hispanics and 42.9 percent of African Americans were uninsured, compared to 23.5 percent of white, non-Hispanics.

What are they going to say about us (QA/Test) 5 or 10 years from now?

Of course, our customers will vote with their feet before it comes to this, but that does not rule out the obituaries ...

The XA/21 bug

santoshz — Thu, 10 Jun 2004 21:50:00 GMT

In the feedback to my last posting, Balaji said

“It has been by personal experience too. If we devote much time and effort to architecture, design, clarifying requirements, attention to detail etc, the outcoming product/feature is usually pretty good with very few bugs.”

No doubt “pretty good with very few bugs” is relative to the application market you are in. Still, I am sure that every tester knows only too well the feeling of someone else finding this impossible bug, and then wondering “Could we have improved out testing methodology so that we caught that one before the egg hit our faces?”

At this point I think the world (deep down aren’t we all testers?) divides into two camps – one which accepts the occasional smell of egg as fact of life, and another that says it is time to learn from past mistakes and re-engineer the process so it can never happen again?

So how do you think the folks at GE Power Systems felt when they finally found the XA/21 bug? Before I tell you, let me give you more detail.

GE Power is no Johnny-come-lately – their annual revenues are in excess of $ 20 billion – and their XA/21 system has over 3 million operational hours (340 years!) since it was first introduced in 1990. So, on August 14th, 2003, when 50 million people in the Northeast United States and Canada lost power, I am guessing that their QA staff in Florida went home pretty unconcerned that their XA/21 energy management and supervisory control and data acquisition (EMS/SCADA) system had any role to play in the ‘perfect storm’ that hit that day.

But by late October, after GE and their energy consultants from KEMA Inc spent weeks going through 4 million of lines of C/C++ code, they had identified the race condition that led to operators at FirstEnergy Corp’s Akron, Ohio control room being in the dark while three of the company's high voltage lines sagged into unkempt trees and tripped. Because the alarm portion of the XA/21 system failed silently, control room operators didn't know they were relying on outdated information; apparently, choosing to trust their systems, they even discounted phone calls warning them about worsening conditions on their grid.

There is an interesting fractal symmetry here, with the XA/21 encountering its own little perfect storm that caused the unknown race condition to be hit, which in its way was an ingredient in the larger storm in North America that day. The drama in The Register’s reporting is not something I normally associate with the software industry.

Sometimes working late into the night and the early hours of the morning, the team pored over the approximately one-million lines of code that comprise the XA/21's Alarm and Event Processing Routine, written in the C and C++ programming languages. Eventually they were able to reproduce the Ohio alarm crash in GE Energy's Florida laboratory, says Mike Unum [manager of commercial solutions at GE Energy]. "It took us a considerable amount of time to go in and reconstruct the events." In the end, they had to slow down the system, injecting deliberate delays in the code while feeding alarm inputs to the program. About eight weeks after the blackout, the bug was unmasked as a particularly subtle incarnation of a common programming error called a "race condition," triggered on August 14th by a perfect storm of events and alarm conditions on the equipment being monitoring. The bug had a window of opportunity measured in milliseconds.

“There was a couple of processes that were in contention for a common data structure, and through a software coding error in one of the application processes, they were both able to get write access to a data structure at the same time," says Unum. "And that corruption lead to the alarm event application getting into an infinite loop and spinning."

After the alarm function crashed in FirstEnergy's controls center, unprocessed events began to cue [sic] up, and within half-an-hour the EMS server hosting the alarm process folded under the burden, according to the blackout report. A backup server kicked-in, but it also failed. By the time FirstEnergy operators figured out what was going on and restarted the necessary systems, hours had passed, and it was too late.

So in which of my camps might GE find itself?

The company did everything it could, says Unum. "We test exhaustively, we test with third parties, and we had in excess of three million online operational hours in which nothing had ever exercised that bug," says Unum. "I'm not sure that more testing would have revealed that. Unfortunately, that's kind of the nature of software... you may never find the problem. I don't think that's unique to control systems or any particular vendor software."

Even if Unum is a manager, I am guessing he has some grease under his fingernails, knows is customers, and has to get a product out the door while maintaining his company’s share value. On the other hand Peter Neumann, a principal scientist at SRI International, can take a more detached, academic, arm’s-length view. (“My main research interests continue to involve security, crypto applications, overall system survivability, reliability, fault tolerance, safety, software-engineering methodology, systems in the large, applications of formal methods, and risk avoidance.”)

[He says] that the root problem is that makers of critical systems aren't availing themselves of a large body of academic research into how to make software bulletproof.

"We keep having these things happen again and again, and we're not learning from our mistakes," says Neumann. "There are many possible problems that can cause massive failures, but they require a certain discipline in the development of software, and in its operation and administration, that we don't seem to find. ... If you go way back to the AT&T collapse of 1990, that was a little software flaw that propagated across the AT&T network. If you go ten years before that you have the ARPAnet collapse.

"Whether it's a race condition, or a bug in a recovery process as in the AT&T case, there's this idea that you can build things that need to be totally robust without really thinking through the design and implementation and all of the things that might go wrong," Neumann says.

Which camp are you in? Or is there a middle ground?

"How do I ..." drive quality upstream?

santoshz — Fri, 04 Jun 2004 18:12:00 GMT

<sheepish grin>Well, my first attempt at something technical fell flat on it face.</sheepish grin> Indranil Banerjee (see feedback) pointed out that the Mathew Wilson’s approach, which I was advocating, effectively killed the symmetry of (a.Equals(b)) == (b.Equals(a)) if one is derived from the other. RIP, unless you can suggest a way around this that does not involve GetType().

I’m going to switch gears here to introduce two subjects close to my heart (for purely personal reasons, as George Tenet said).

The first relates to documentation. One of my pet peeves is that our (Microsoft’s) products are not used to their full potential because we don’t do a good enough job of educating users about 'features' - features that could be useful to them if only they knew about them and how to use them. This is, of course, not at all easy, and is one of those things differentiating experts from novices. Experts know where to find appropriate information and they absorb it much quicker than novices do. Thus they learn faster.

Software is an interesting product to provide documentation support for. Firstly, the way that developers acquire new information and skills is very different – you don’t usually see a builder opening up a product manual while sitting on a half-framed roof, or surgeons hitting F1 in the middle of a tricky procedure. Secondly, experts and novices use pretty much the same version of the product, notwithstanding any industrial and consumer SKU’s

One piece in this big puzzle is the revered FAQ. We have one on C# at MSDN and are always looking for appropriate questions to add to that list. Leave your suggestions in the feedback here, or there.

The other subject relates to making Quality Assurance (QA) more effective and efficient. (Now, if you think that good documentation for a wide variety of developers is an easy challenge, try this one! The world will thank you.)

A phrase that one increasingly hears in testing corridors is “moving quality upstream.” One indication of this is how we increasingly call ourselves QA instead of Test. In the back page column of the first issue* of Software Test & Peformance – a magazine that plans to devote itself to ‘getting it right the first time' - Adam Kolawa (CEO Parasoft Corp) goes as far as to say that we really need to start over from scratch. His basic thesis is that, instead of devoting ever-increasing resources to finding and removing errors from the product (testing-out bugs), we need to reinvent the software lifecycle to prevent errors from entering it.

“This belief that testing can create quality software systems is a fundamental problem in the software industry. We don’t think of the whole process of building and deploying software in a way that would prevent errors, because we don’t believe that it can actually be done. Yet this error-prevention approach is not only possible but necessary. Every mature industry has a already figured this out and stopped relying on testing as a way to make products work. We continue to have faith that testing will deliver quality – but it never does.”

Of course Mr Kolawa would say that, considering he has a product to sell. But what do you think?

[*Issue 1 of ST&P is available as a pdf download. If you go that far, do read the editorial too.]

‘Optimizing’ over-rides of object.Equals and overloads of operator ==

santoshz — Tue, 01 Jun 2004 16:28:00 GMT

For the past year, as the whole world and its dog have blogged themselves onto the NY Times bestseller list (and in many cases off it too), I have gazed at my navel, and wondered if the lint there is the same color the world over.

Then today, I read Mathew Wilson’s piece “Identity & Equality In .NET” (Dr Dobb’s Journal Windows supplement, June 2004) . Which caused me to sit up, hastily button my shirt (yes some of us at Microsoft do wear real shirts) and seek out a blog-editor – just so I could summarize what Mathew said for all non DDJ readers.

Like most (all?) C# programmers, I’ve undergone the baptism ritual of over-riding object.Equals. Having done that once, I now keep Jay Bazuzi’s Equals-in-a-box in a safe place (alongside Kevin’s addition) [See the full list of guidelines at MSDN, including the guidance for operator ==]

Here’s what Jay gives to those who say the magic word (It is pretty much the same as the code sample on MSDN - the Visual Studio formatting is just free publicity.)

// override object.Equals

public override bool Equals(object obj)

    if (obj == null || GetType() != obj.GetType())

            return false;

    // TODO: write your implementation of Equals() here.

    throw new System.NotImplementedException();

    return base.Equals(obj);

And for completeness here is operator == and GetHashCode()

bool operator ==(SomeType o1, SomeType o2)

    return Object.Equals(o1, o2);

// override object.GetHashCode

public override int GetHashCode()

    // TODO: write your implementation of GetHashCode() here.

    throw new System.NotImplementedException();

    return base.GetHashCode();

These are pretty much the canonical forms of the overrides/overloads

Now, if you expect that you are going to be testing equality a bunch of times (sorry for the jargon), you may be tempted to see how fast you can make your code. [Sadly, omerta prevents me from telling you the real reason why developers flog their code to the extreme.]

So, starting from this point, Mathew (who seems to have a penchant for seeing how fast something will go) goes on to systematically derive his optimal versions. He then did some simple performance comparisons. It was the performance differences that caused me to sit up so sharply.

In Jay’s over-ride of object.Equals(), above, if you replace the calls to GetType() with our own beloved as operator, like so

//if (obj == null || GetType() != obj.GetType())

if (obj == null || Object.ReferenceEquals(obj as SomeType, null))

        return false;

Mathew says you can knock off about 60% of the original cost.

Now admittedly the semantics of this version is modified ever so slightly, as the as will return non-null if obj is derived from SomeType. However, typically that is what you want, isn’t it.

I’ll quickly summarize the rest of Mathews findings just to underline the significance of the 60%.

Replacing

        Object.ReferenceEquals(obj as SomeType, null)

with

        ((Object) obj as SomeType)==null)

netted about 4%.

And all the effort of working around the virtual call to Equals in the operator == overload, and narrowly surviving two potential gotcha’s, to end up at

public static bool operator ==(SomeType o1, SomeType o2)

    //return Object.Equals(o1, o2);

    return (Object)o1 == null ? (Object)o2 == null : o1.Equals(o2);

Shaved off just 2.5%. Hardly worth the potential errors.

Ordering the member comparisons with fastest comparisons first (in Mathews case ints before strings) took off a respectable 18%.

All told, Mathew was able to knock off 80% from his original canonical form.

Just to convince myself, I ran his suite on both the Everett and Whidbey Community Preview versions of Visual Studio. In my case, as knocked off only 27 to 37% of the execution time, while reordering member comparisons contributed 25 to 39%. In both versions, combining all the tweaks reduced execution time by 65% (YMMV - drive safely, wear your seatbelt, and remember to override GetHashCode).

All this give me new respect for the isinst MSIL instruction, and underlines something I hear time and again. Which is “beware of premature optimization, and don’t optimize until you know where the bottleneck/s are.”

What do you want to do today?

santoshz — Thu, 20 May 2004 20:08:00 GMT

Today is the first anniversary of the day I started working at Microsoft – in the C# compiler quality assurance team. After 21 straight years in Universities, either as a student or in the lab, the past year has been quite a change.

It’s a great team, and a nice company - “Where do you want to go today?”

A year is but a tick of the cosmic wheel. About 35 years ago, when I was barely 4, on July 20, 1969 the lunar module Eagle landed on the surface of the moon, at Tranquility Base. Aldrin and Armstrong were there less than 24 hours, but they left their mark, literally and figuratively; their footprints in the lunar soil are expected to last millions of years.

Back on earth, and 3.75 million years earlier, two other individuals - one smaller than the other - took maybe 30 seconds to leave their own trail in the moist volcanic ash near the village of Laetoli in the eastern Serengeti Plain. Not quite 10 years after television brought the images of Apollo 11 home, Mary Leakey excavated a 75 foot long trail that included these footprints . (See Scientific American September 1998 for a good article). Although not yet fully bipedal, the shape of the early hominid foot is already uncannily similar to the modern human (on right above). After spending over 5 decades of their professional careers at Olduvai Gorge and Laetoli in East Africa’s Rift Valley, Mary and Louis Leakey too left their mark on our understanding of where we came from.

So what’s the connection between Laetoli, Tranquility, and Redmond? Or between Leaky, Armstrong, and myself? Or between a hand pick, a spacecraft, and a piece of software? To me it’s in the images.

Imagine now that it is late evening, and you - that juvenile hominid above - are returning from a seasonal watering hole to the safety of the trees at the edge of the savannah. As you walk hand-in-hand with your parent, the full moon rises ahead of you, and both of you pause to gaze at it in awe.

Dare you imagine where you want to go today? What footprints will you leave in the sands of time?