Buggin' My Life Away : Mac BU

Craig's Bug

Rick Schaut — Fri, 01 Feb 2008 05:01:00 GMT

One of the things I love about this job is tracking downinteresting bugs. It’s like readinga good who-done-it, complete with plot twists, multiple potential suspects and a fascinating array of characters. Late last September, Craig Eisler submitted a bug that had all of these elements.

Now, when your general manager submits a bug, you don’t necessarily have to get on it right away. But, it’s not a good idea to let it linger for too long. So, after this bug had been on the list for about a week or so, I decided it was time to figure out what was up.

The bug involved a document Craig had been editing with revision marking turned on. He’d made some changes to the document, saved it, quit Word and, when he opened the document back up, some of the spaces between words were missing.

With a bug like this, the first thing I do is open it up in Win Word. If the problem appears there, then there’s a strong likelihood that the bug is in the code that writes the file. If the bug doesn’t appear in Win Word, then I know to start looking at the code that reads files. In this case, the problem appeared in Win Word, so my first suspect is going to be somewhere in the save code.

Also, the missing spaces seemed to be in or around areas of the document where revisions had been made. So, the first suspect was somewhere in the code that writes revisions. I created a document, added some text, turned on revisions made some changes and saved the file stepping through the code that handles revision marking.

Nothing. Everystep along the way, everything was happy. The code was writing all the appropriate XML tags, and none of the text was missing. Of course those two sentences summarize about two-three hours of sleuthing, after which I was able to exclude the revision marking save code as a suspect.

Well, maybe it’s something that only appears to be related to revision marking. Let’s take two files, one that exhibits the problem and one that doesn’t, and compare the contents (much easier to do now that the document content is XML).

Again, nothing. Other than a few gratuitous differences (revision save ID’s, for example, which are uniquely generated every time new text is saved in a document), everything in and around the missing spaces was identical. Another couple of hours spent cracking open various documents and comparing the XML, and I still haven’t a clue whothe real culprit is.

Well, but, if the documents are identical, maybe the problem happens when we read the file. So, I set a breakpoint on the code that handles runs of text from the XML parser, and, sure enough, our “characters” callback wasn’t getting called for just the missing spaces. Hm… Progress.

About another hour or so, I had tracked it down to some general XML parsing code we’d ported from Win Office. One of the problems with writing filing code that’s intended to handle future situations is that, when you write the code, the future hasn’t arrived yet. Well, for Win Office, Mac Office represented the future, and some of the stuff we’d added exposed a subtle bug in the code we’d ported (note: the problem has been fixed in WinOffice before Mac Office shipped, so people shouldn’t run into this at all).

By the time I had finished all of this, implemented a fix and verified the fix, it was about 1:30 in the morning. When I get engrossed in a good mystery, I sometimes find it difficult to put the book down. Same thing goes with good mystery bugs.

And, the bug being a good mystery bug that geeks like Craig and I appreciate, I sent off an e-mail to Craig outlining the basic nature of the bug and the fix (which turned out to be just a couple lines of code—this is almost always the case with a good mystery bug). This being in the middle of the FIFA Women’s World Cup, I ended the mail by saying that I was going to go home, drink some coffee, and stay up to catch the USA match (I forget who the opponent was).

The next day, I got a response from Craig with some questions about the bug. I replied to Craig’s questions and, at the end of the e-mail, lamented over the fact that, after all that effort and all that coffee to stay up to watch a soccer match, our local cable company saw fit to pre-empt the last 30 minutes of the match with a taped question-and-answer program starring Mike Holmgren, the Seahawks coach.

For a born-in-Green-Bay, prick-me-and-I-bleed-green-and-gold, Packers fan like me, having a soccer match for which I’d stayed up all night to be able to watch be preempted by Mike Holmgren is seriously rubbing salt into the wound, and I conveyed my displeasure at this to Craig by saying that the only reason we still have a working television in the house is the fact that there were people asleep in the house at that time.

What ensued was a series of e-mails between Craig and I with a dual thread about the bug he’d reported and my television set. His response to my previous e-mail was to suggest that I should have taken the TV outside and smashed it there. Wake up the neighbors instead of the kids, and I get to vent my hostilities.

In my reply, I explained that this was not an option giventhe size (36”) and weight (over 260 lbs) of my TV set, to which Craig replied, “A CRT!? How quaint! Do you get slivers when you go to the bathroom after watching that TV?”

Having an uber-geek general manager is cool. You get great bug reports, and he likes to hear about the nature of the bug and the fix. Just be a little careful when you talk about the technology around your house.

Rick

Currently playing in iTunes: New World Blues by Gov't Mule

The GM Shuffle

Rick Schaut — Sat, 09 Jun 2007 20:21:00 GMT

By now you’ve read the news. MacBU has a new general manager. I only found out about this via e-mail on Thursday, and, yesterday, I got a chance to meet Craig. I’ll get to my initial impressions of Craig in a bit, but I also want to talk a bit about Roz Ho.

Before I talk about Roz, I should point out that, in Microsoft terms, I’m a wierdo. That’s not just because I work on Mac software. No, I’m an anomoly in a very different sense. In a couple of weeks, I’ll mark my 17th anniversary of working on various versions of Word, all but one of them for the Macintosh. That’s so rare, that people have suggested they make a place for me somewhere in the Microsoft Museum (just what I want: to become a museum piece).

Microsoft is pretty big on career advancement. It’s not a “move up or move out” kind of thing. It’s more of a do something cool and good here, you get to do something cool and good somewhere else. That’s why I’m such a freak. I keep managing to advance my career without having to move on to something completely different.

For a general manager, this career advancement thing becomes something of a problem. Opportunities to climb the management ladder start getting really thin. So, how do you reward a GM for doing a great job? You find a different kind of business for that GM to run—something that might involve cool new technologies, or something that requires a different approach to management, or a host of other kinds of chances to do something, well, different.

So, earlier this year, a very interesting and exciting opportunity opened up for Roz to advance her career. This kind of thing always brings out mixed emotions in me. I hate losing someone as good as Roz, but I’m also proud that someone from MacBU is being recognized for doing a great job. I’m very excited for Roz in her new role, and I think it will be good for both her and for Microsoft.

Roz became the general manager of MacBU back in December of 2002. She oversaw the release of what has been the most successful version of Mac Office ever. Her more than four years as general manager covers nearly half of Mac BU’s ten year existence. Of the three general manager’s I’ve worked with until now, I think Roz has been the best, and she’s certainly been the most pleasant to work with (which is, by no means, a knock on either Ben Waldman or Kevin Browne).

I say all of this to express my personal gratitude to Roz for what she’s done for MacBU, for the Macintosh in general, and for me personally. I’ll miss you Roz, though I still think you messed up the Connectix deal for not getting all of us our own “Switch/Switch Back” t-shirts.

So, what about Craig Eisler? As I said, I only found out about this on Thursday, but I see that he’s a blogger and enough of an Apple enthusiast as to have 5 Apple TV’s, 3 iMacs, a Mac Mini, a Mac Cube and a PowerMac G4. Oh, and, ah, I think there’s a video iPod in the mix as well. So far, so good.

I did a bit of poking around the internet, and found quite a bit of information about Craig, like he was once a competitive power lifter, and he once, quite by accident, managed to bring down the entire Canadian banking system. Four times. In one day.

UWTV did this interview back in April of 2004. It’s a bit long, but very interesting.

On Friday, we had a bit of a get-together out on the patio outside our building, and I had a chance to chat with Craig. If I had to choose a single word to describe my first impressions, it would be “exuberant”—almost, but not quite, effusive. We swapped a few stories. Someone mentioned my suggestion box, and he said that he’d read my blog and thought the suggestion box was a scream.

So, he gets the Mac, he gets Microsoft and he understands that running a successful business is measured in terms of the value of the product to users and customers. Best of all, he gets my twisted sense of humor. I’ll have to try out some of the Canadian jokes my daughter brings back from school. C-eh? N-eh? D-eh?

Rick

Currently playing in iTunes: You da Mann, by the Derek Trucks Band

There Comes a Time

Rick Schaut — Thu, 05 Apr 2007 04:48:00 GMT

In every project, there's a point where you have to stop tweaking this and touching up that little bit of behavior and focus on just fixing bugs. If you don't, then you never ship. Nadyne gave some of the details over on mac mojo.

Well, we're getting close to that time with Mac Office 12. In honor of nearing such a milestone, I prepared something for our program management friends. These are the folks who are responsible for designing how the various features are supposed to behave from the user's point of view. It's a suggestion box, and it looks like this:

Mind you, this suggestion box is for PGM use only. Think of it as my way of saying that now is the time for us to shut up and listen to the real users who are trying out the product during the private beta.

Enough fun. I need to get back to fixing bugs.

Rick

Currently playing in iTunes: Beautifully Broken by Gov't Mule (w/ George Porter Jr.)

Lounging Around at MacWorld

Rick Schaut — Thu, 11 Jan 2007 02:18:00 GMT

MacWorld always has its surprises, but some of them are quite personal. For me, the surprise was the discovery, at 5:20 this morning, that a previous occupant of my hotel room had set the alarm-clock but hadn't turned off the alarm. I've been told by others that my failure to check the alarm clock when I checked in is a sign that I don't attend trade shows often enough. Actually, I think I probably attend too many as it is.

Whatever the case, thanks to this inconsiderate person, the Word file format converter is now about four hours closer to being done than it would have been had said person learned how to turn the alarm clock off. I expect, however, that those hours will revert back during an upcoming, afternoon nap.

I got a chance to chat with John Welch today, and I'm happy to inform you all that he appears to be in much better shape than he did yesterday. And, you know, things were so busy in the Blogger Lounge, that I forgot to ask John if he's been able to check out Songlines by the Derek Trucks Band.

I've yet to get any opportunity to personally wish Shawn and Lesa a happy wedding anniversary, so I will here. Happy anniversary, Shawn and Lesa. In one of those cute MacWorld stories, Shawn and Lesa met at MacWorld three years ago. Two years ago, Shawn proposed to Lesa at MacWorld, and, last year, they got married at MacWorld. I think we should petition the event sponsors to rename MacWorld the Shawn and Lesa Wedding Anniversary Celebration.

Shawn does have a lot of pictures from the show, and a couple of cool interviews with Chuck Leavell and Bob Weir. It seems MacWorld always has at least one musician I admire roaming the floor, but I still think my personal best was getting Herbie Hancock's autograph in the Microsoft booth a few years back.

Lastly, I almost got a chance to chat with Andy Ihnatko. I still remember Andy raking me over the coals about Mac Word 6. I hadn't realized how much shorter than me Andy is, and that's saying quite a lot, but I still think Andy is much taller in stature. He's still one of the best writers in the Mac commentary field.

That's about all from the Blogger Lounge. I need to bag some z's before it's time to spend an evening with the Mac MVP's.

Rick

Currently playing in iTunes: Midnight Pass by Sea Level

Open XML Converters for Mac Office

Rick Schaut — Thu, 07 Dec 2006 21:04:00 GMT

There’s been a bit of flak about the Office Open XML file format converters for Mac Office. Sheridan posted an update on MacMojo, and Schwieb weighed in regarding some of the comments that people have made. There’s quite a bit of speculation gong on, and not a whole lot of information, so I’m going to try to dispel some of the fog.

This discussion centers on Word, because I’m a Word developer, but the general ideas hold for all three of the affected Office applications. The most significant difference between Word and the rest of the suite is that Word has a converter API. There’s a WinWord converter SDK that’s downloadable from the Microsoft support web site. While there are some subtle differences (FSRef’s instead of file paths, for example), the overall API is the same for Mac Word. Of particular importance is the fact that the lingua franca for converting Word files formats is RTF.

So, in order to write a converter for Word, you need two things: 1) a component that reads and writes the external file format; and 2) a component that generates and parses RTF. Also, because of the hierarchical structure of XML, you need to have some form of intermediate representation of the file.

Let’s put our Win Word hat on for a second, go back in time about two years, and think about how we might do this. Well, by the time Win Office ships, we’ll have a software component that satisfies all of those needs: Word itself, or the new version of Word, to be precise. So, one, very efficient, way to implement that converter is to refactor the UI out of Word 12, repackage the result up with any other necessary components, and write a wrapper around all of it that exposes the API that the older version of Word expects converters to implement.

Do that, and you can ship the converters the same time you ship Office. The big upside of this idea is that you can really narrow down the scope of testing you do on the converter itself, because you’ve already tested both the RTF and the Open XML components by testing Word itself. So, you gain leverage from both a development and a testing perspective.

Now, let’s put our Mac Word hat back on, and think of what our options are given the reasoning I’ve stated above. You can’t really ask the Win Office team to toss their idea in the trash just so you can work on the converters in tandem. Well, you can, but one would have to be very optimistic to expect more than a polite, “Sorry.” I can’t think of a clearer example of the tail trying to wag the dog.

Instead of following in Win Word’s footsteps, how about we spin off a separate development team to work on the converters separately from Word itself? I've read suggestions made by some that writing converters from scratch could have been done in a relatively (in some cases ridiculously) short amount of time. So, let's test that idea by doing some back-of-the envelope calculations. You can check these numbers for yourself by downloading the reference XML schemas and performing some searches through the .xsd files.

First, when could we have realistically started working on this? Well, not before Office 2004 shipped in April of 2004, so, ignoring the availability of specifications for the new format, let's assume that we began work on this roughly two years ago. The final draft of the spec wasn't submitted to the ECMA until this past October, so in terms of actually having a spec to write to, 24 months is extremely optimistic for the time period available.

How big is the task? Word, alone, has more than 1100 individual XML elements that need to be processed. We do this processing by writing something called a "handler", and each one of these elements needs a handler.

Now, some of these elements are more complex than others. A single, user-defined document property isn't very complex. A paragraph, or a document section, can be very complex. For some of these handlers, one developer can whip out two or three a day. Some of the other handlers will take a single developer up to an entire month to complete. Trying to get more than one developer working on the same handler at the same time ends up being very counter-productive. So, one handler per developer, and, on average, it's fair to assume productivity of one handler per dev per day.

At that rate, a team of 5 developers will implement 25 handlers a week, which means that we'd have all the XML handlers written in 44 weeks. Well, a little more than that, because I've rounded the number of elements down to the nearest 100. Nevertheless, we’ve taken a little less than a year to get the converters reading the new file format. We still aren't writing the new file format, we have the RTF side of things to worry about, which is actually more complex than the XML side, and I’ve completely left out all of the design and coding for the intermediate representation of the file. The intermediate representation, itself, is at least 6 to 8 months worth of work.

In other words, we're almost halfway through the schedule, with less than a quarter of the development work done. You want more developers? I don't have more developers. This is just for Word. We need additional teams for Excel and PowerPoint. People want Universal Binaries of Mac Office in their hands, they’re adding new features to Win Office 12 that Mac Office 2004 won’t understand, Apple has a new HIView architecture that requires some re-architecting of parts of Mac Office, and none of this work adds a single new feature to Mac Office.

More importantly, we’ve also run out of time to test the converters. Had we started writing converters from scratch, by the time we had something fully tested and ready for public consumption, it would have taken us longer than it has taken us on the route we’ve chosen, in no small part due to the fact that the current route we’ve chosen allows us to leverage almost all of the development work of the Win Office team.

The only reasonable choice for Mac Word has been to follow in Win Word’s footsteps. For those of you who attended the last Mac BU customer council meeting in Redmond and were wondering what I was doing while sitting in the back corner, now you know. I was busy refactoring Mac Word so that Mac Word 12 could, eventually, become the converter for the new file formats.

The big win for this strategy is that we get to do all of the things that customers are asking us to do with the next version of Mac Office: Universal Binaries, support for most of the new data types in Win Office 12, re-architecting the UI to take advantage of composited HIViews and add some compelling new features.

Lastly, can we port the Win Word converter? Well, actually, in a way, porting the Win Word converter is exactly what we have been doing, but we’re still faced with having to wait until Win Word ships before we have the final source code to merge into what we’ve already ported. Once that merge is done, then we still have to go through several months’ worth of testing and bug fixing before they’re ready for public use.

And that is precisely why there’s a delta between Win Office 2007 shipping and the full availability of converters for Mac Office.

Update: I’d like to clear up some things about what I said earlier. My back-of-the-envelope estimates included a lot more work than just supporting Open XML in Mac Office. Open XML is the easy part. It included the work required to generate RTF in both directions and to implement tools for developers.

If we had to add support for Open XML to Mac Word 12 without being able to port code from Win Word, the read/write estimates shrinks down to about 8.5 man/years (44 weeks x 5 devs x 2 for read+write). As I recall, this about half of what it took to add HTML support to Word: 10 or so devs over a release cycle of 2 years. Doing the work for PPT and Excel isn’t strictly a multiple of Word, because about 30% of the XML elements are shared between the three apps. So, for all of Mac Office, I’d estimate it would take a total of about 5 devs over the release cycle to add full Open XML support starting from scratch, as part of the larger project.

Rick

Currently playing in iTunes: Time Loves a Hero by Little Feat

Muddy Mojo Waters

Rick Schaut — Mon, 28 Aug 2006 23:26:00 GMT

Somewhere in the backroads of Issaquena County, Mississippi, along the banks of the Mississippi river, McKinley Morganfield was born. Likely due to his early life so close to the Mississippi river, McKinley later came to be known as Muddy Waters. The father of Chicago blues. The man most responsible for introducing the electric guitar into the genre. At one ponint in history, the terms "Electric blues" and "Chicago blues" were synonymous.

Talking about Muddy Waters, Eric Clapton once said, "His music changed my life, and whether you know it or not, and like it or not, it probably changed yours too." The very roots of Rock-n-Roll itself traces directly back to the music of Muddy Waters. His influence is comparable to, nay even surpasses, that of say Charlier Parker or Dizzy Gillespie on Jazz. So wide is Muddy Waters' influence that a long standing British combo and a well-known monthly magazine, as well as a Bob Dylan tribute, took their names from the title of one of his songs, "Rollin' Stone."

But I'm not going to talk about Muddy Waters' music per se. Rather, Muddy Waters is known for another contribution to American culture. More than anyone else, Muddy Waters is responsible for the word "mojo" having made its way into our lexicon.

I say that with some care. His contribution derived not from his skill as a lyricist, but the strength of his performances. Nonetheless, the lyrics of many songs Waters' made popular were suffused with references to African folk magic. The song "Hoochie Coochie Man," written by Waters' bassist, Willie Dixon, is typical:

I got the black cat bone, I got a mojo tooth
I got the John the Conqueror root, I'm gonna mess with you

Another classic is Preston Foster's, "I've Got My Mojo Working," which opens with the line, "I got my mojo working, but it just don't work on you."

So, back on March 29, 2005, Dave Winer was in some kind of Muddy Waters, mojo mood mostly involving Yahoo 360. Coincidentally, Betsy Aoki shared her Got Dot Net Blues with us, which prompted me to add this comment.

Fastfoward to a couple of weeks ago in a meeting where we'd invited Betsy to share some thoughts on running our Mac Office Team blog. We'd had a name for it, but hadn't really settled on that name. During the course of the discussion with Betsy, however, she used the phrase, "Your Mac mojo," in a rather off-handed, throw-away kind of fashion. The blog's name itself wasn't the topic of disussion, but, in one of those unpredictable, synergistic moments, the name stuck. So, we now have our Mac Mojo woikin'.

Did my comment have anything to do with Betsy's use of the phrase "Mac mojo" during our meeting? I haven't a clue. But, hey, I'm allowed a certain amount of room for dellusion over such things. Whatever the case, I hope everyone enjoys the new team blog. There are some pretty interestings posts in the queue.

Rick

Currently playing in iTunes: Hoochie Coochie Man by Eric Claption

Virtual PC and Visual Basic

Rick Schaut — Wed, 09 Aug 2006 21:01:00 GMT

Scene: A fly-over view of Microsoft's campus focuses in on the building that houses Mac BU. The camera "flight" enters one of the offices through a window. During the fly-over, a voice-over says:

New 3.0 GHz Mac Pro: $3,599.00
Two 23" Apple Cinema HD Displays: $1998.00
Office space: $350.00/mo
Competent developer...

You get the picture. For years, one of our program managers has been coming into my office with feature questions that invariably begin with, "Is it possible to...?" Well, of course it's possible. Anything's possible if you give me enough time.

So, earlier this week, we announced that we're not going to ship a Universal version of Virtual PC and that we're going to discontinue support for Visual Basic for Applications. The latter is particularly painful for a number of our customers, and people are quite understandably livid about it.

While Erik Schwiebert has posted a sound explanation of the technical hurdles involved, there remains the more general question of finding the resources capable of doing the work required to overcome those hurdles. Many people who are not at all familiar with the exigencies of software development look at Microsoft's balance sheet, and the maxim that anything's possible in this industry, and wonder whether or not we're pulling their legs. They don't understand that the constraining resource isn't money. It's people.

There are two pieces in the people puzzle. The first is head count, and, for various reasons, Mac BU has always operated with open head count. "Open head count," means having a budgeted slot to hire someone, but not having an actual employee in that budgeted slot. Due to the turn-over of people who move on to new challenges, and the difficulty of finding candidates who we believe won't write something worthy of being posted to The Daily WTF, despite the steady stream we keep interviewing, I don't see this condition changing in the near future.

The fact that we've always had open head count points out a subtle aspect of the budgeting issue. Picture yourself being Roz Ho's boss. She comes to you asking for an increased budget for more head count. You look at the existing budget, and see that there are open head count. Your question is likely to be, "Why are you asking for more head count when you haven't filled the head count you already have?" The point: the people resource issue isn't a budgeting problem.

The second part of the people issue is a bit more difficult to understand when you haven't spent a great deal of time trying to ship software in the real world, but there is a point where adding new people to a project results in diminishing returns. Fred Brooks refers to this as the Mythical Man Month. If you ask me whether Mac BU is still on the increasing side of the curve or if we've come close to the diminishing side, I'd honestly have to say that I really don't know. This is one of those cases where you don't really know you've hit the point of diminishing returns until you get there, and, even then, recognizing that that you have is exceedingly difficult. There are a number of variables that affect the time it takes to complete a software project, and it's very difficult to account for them all.

However, Schwieb's estimate of two or more years to rework VB/VBA isn't far off the mark, and it assumes that we put every resource onto that project that we can possibly put on it. That estimate also takes into account the issue of dependencies. This part of the Mythical Man Month isn't all that well discussed in the Wikipedia article linked above, but it's one of the more important problems we face. The number of people you can effectively put on a project is determined by the number of discrete tasks into which the project can be broken down and by the extent to which those discrete tasks depend on the completion of other tasks.

Consider the VBA execution engine that Schwieb mentioned. One might be tempted to take each opcode, and turn that into a discrete task to be worked on by an individual developer. Unfortunately, that kind of approach will lead to less coherence in the overall project. Are there any common elements to certain subsets of the full set of opcodes? If there are, and we have one developer per opcode, then there's a high probability we'll have a large number of subtly different implementations of these common elements. The more discrete tasks into which you break up a project, the greater the logistical difficulty of integrating each of those discrete tasks into a coherent whole.

Now, I've read a number of suggestions that people have made: take a Ruby on Rails approach, figure out a way to use PowerPC/CFM VBA running under Rosetta, and a few others I can't recall off the top of my head. There are ideas we'd explored that I haven't seen anyone suggest. I can't think of any one of them that would not have involved more work than working with the design parameters of the current VBA implementation--not the least of which is the fact that all of the applications implement their object models using the OLE Automation, dual-dispatch/type library approach. One has to be very careful with such ideas, because there are a number of hidden gotcha's that are very difficult to accurately assess.

So, could we have done something about VB/VBA? Certainly, if we'd had enough time. Do we make those users who really don't care about VB/VBA wait several years before we ship a Universal Binary version of Mac Office? At some point, the tail starts to wag the dog.

But that does not mean we are unaware of the pain this will cause to a significant number of users. If you think we are not aware of that pain, consider this. David Weiss can give you a better number on this, but our testing methodology has always made extensive use of scripts for automated tests. Not just a couple of scripts, but thousands of scripts per Office application. At one point, all of those scripts were written in VB/VBA. In order to carry that testing effort forward into the era of Universal Binaries, every single one of those scripts had to be rewritten in AppleScript. I don't think it's even a remote exaggeration to say that our use of VB/VBA was at least a couple orders of magnitude greater than even our most automated customers. Do we know your pain? You bet we do.

Rick

Currently playing in iTunes: Feel So Bad by The Derek Trucks Band

What's Wrong with this Code: The Answer

Rick Schaut — Wed, 14 Jun 2006 18:21:00 GMT

Bjarne Stroustrup is a brilliant man, which means he's very good at defining programming languages. It also means he's not very good at writing books about the programming languages he's defined. His writing goal is concise precision. Unfortunately, for those of us who have to read what he writes, that quest for concise precision often translates into some of the driest reading in existence.

Yesterday, I asked What's Wrong with this Code? Unfortunately, the answer to that question involves one of the driest sentences in Stroustrup's book on C++. In fact, it's the very first sentence of section 6.2.2 of The C++ Programming Language:

The order of evaluation of subexpressions within an expression is undefined.

Even as I write this, I can hear the collective, "Huh?" So, let's take at look at the key line of code in yesterday's example. It's in the function, Foo():

_val += Bar(val);

This is an expression with subexpressions. It's a contractional form of the expression:

_val = _val + Bar(val);

It's rather like the contraction "it's"--a way of saying "it is" using fewer letters and syllables. The subexpressions are the bits on either side of the + sign and the = sign.

Now, I'd like to translate that expression back into English for you, but I can't. At least I can't give you a definitive translation. The expression is ambiguous. We can read it as:

Evaluate the function, Bar(val), add the result to the value in _val, and place the sum back into _val.

Or, we can read it as:

Take the value that's in _val, add it to the result of the function, Bar(val), and place the sum back into _val.

Thing is, the actual result of the code example I gave will be different depending upon which interpretation we use. That's because Bar() changes _val (programmers call this a subexpression side-effect). If we choose the first interpretation, then the result of yesterday's program will leave _val with a value of 2. If we take the second interpretation, then the result of that code will leave _val with the value of 43.

Well, that's fine, Rick, but what does this have to do with what Stroustrup said? Stroustrup is saying that compiler writers are free to choose either of these interpretations that I've given above. In fact, compiler writers are free to choose one interpretation at one line of code, and another interpretation in a different instance of the same line of code. While this sometimes happens, compiler writers tend to choose one interpretation and stick to it.

And that's where the real problem lies. Remember, this is a contrived example. Given this kind of contrived coding example, most programmers worth half their paychecks can spot the problem a mile away. But, real-world software models real-world problems. And, real-world problems are complex. So, in the real world, someone writes Bar() without the line of code that modifies _val. Some time later, someone else adds Foo() which uses the computation done in Bar(). A little time after that, someone changes Bar() so that it now modifies _val.

All of these changes get checked into the code base, built into the product, and tested. They might even get tested over and over again. We might even ship several versions of the product, without ever running into any problem that any user will see. And, so long as the compiler we're using takes the "correct" interpretation, there won't be any problem in the code that we actually ship.

Then someone comes along and forces us to start using a new compiler, like when Apple announced the move to Intel processors. Should the new compiler interpret these expressions differently than the way the old compiler interprets these expressions, then we'll have a few subtle bugs in our code.

Now, I've read articles where people describe this programming style as "Microsoft's patchwork programming style," or some such similar--as if the word "patchwork" accurately describes incremental development with modular programming and as if Microsoft is the only entity in existence that uses these programming techniques. Actually, everyone does it this way, and any group engaged in trying to write programs to solve complex problems will end up with some code like this in their products. It's inevitable. I'm quite certain that Adobe has encountered a handful of these kinds of issues in their code, and I'll bet that Apple had to deal with a few of them as well.

Worse yet, in all of the compiler errors and warnings that Erik Schwiebert mentioned in the post that prompted me to offer this little programming example, not one of them involved problems of this type. Those of you who have XCode installed on your machines, copy and paste the code from my example into a test file, and compile it using GCC with every warning you can think of turned on. You'll get nary a peep out of GCC.

We know that many people are anxiously awaiting universal binaries for Office and Adobe's products. Please, be patient. You really do want us to find all of these subtle issues before we ship these products to you, and that takes time--mostly testing time even when you have a lab like this one where you can run those tests.

Kudos to Doug for being the first to say where the problem lies, and kudos to Joshua Ochs for being the first to quote the correct section number from Stroustrup. And, Joshua's guess was almost correct. CodeWarrior uses the first interpretation above. GCC uses the second.

Update: Upon reflection, I thought it worth adding, for some of the more experienced programmers out there, some further details that indicate just how insidious this can be. Imagine some code that began its life as Java. Later, it got ported to C++. The key part of the body of Foo() is a tight loop that terminates when _val reaches 0, and Bar() is really invoked through a pointer to member function in an array of pointers to member functions indexed by _val. This is the kind of stuff that keeps me awake at night.

Rick

Currently playing in iTunes: Everyday (I Have the Blues) by The Marshall Tucker Band

What's Wrong with this Code?

Rick Schaut — Tue, 13 Jun 2006 22:44:00 GMT

A while back, Erik Schwiebert wrote about some of the travails of moving our build system from one using CodeWarrior to one using XCode and GCC. Erik mentioned the huge number of errors and warnings we've had to resolve, but, in reading some of the reactions around the web, I'm afraid that some people, particularly people who know nothing about compilers and developing software, seem to have completely mistaken Erik's remarks.

Someday, I'll understand why people are motivated to render opinions on matters of which they are entirely ignorant, but, for now, I'll have to satisfy myself by offering some remedial education. I'll start by asking, what's wrong with the following code:


class FooBar
{
public:
    FooBar(int val): _val(val) {};
    ~FooBar() {};

    void Foo(int val)
        {
        _val += Bar(val);
        };

    int Val() { return _val; };

private:
    int Bar(int val)
        {
        _val = val;
        return _val;
        };

    int _val;
};

int main()
{
    FooBar fb(42);
    fb.Foo(1);

    int val = fb.Val();

    return 0;
}

Some of my readers will spot the problem immediately. To those who do, I'll remind you that this is a contrived example. A real-world example will have a class that has 10 or more methods, and none of those methods will be as trivial as Foo or Bar.

Update: Fixed code problems due to .Text's handling of HTML entities.

Rick

Currently playing in iTunes: One Way Out by The Allman Brothers

Mac Office 2004 Resource Kit

Rick Schaut — Fri, 09 Jun 2006 07:17:00 GMT

Looks like we've updated the Mac Office Resource Kit to Version 2. The Resource Kit is a guide for system administrators and other IT professionals who manage and support Office on a network. We have other deployment documentation that can be found on the main Resource Kit page on MacTopia. Details on the Resource Kit for Mac Office 2004 can be found here.

Some key changes between Version 1 and Version 2 are:

A major new section that guides the system administrator through the Office 2004 deployment process, taking advantage of standard Apple software such as Workgroup Manager, Apple Remote Desktop, and PackageMaker. Topics include:
- Creating a custom Office 2004 installation
- Procedures for several different methods of Office deployment
- Managing preferences on user computers
Topics about using Office 2004 with related Microsoft products, including:
- Expanded information about working with Messenger for Mac and Live Communications Server
- New information about working with SharePoint, Internet Security and Acceleration (ISA) Server, and Open Database Connectivity (ODBC) and SQL Server
Expanded information about configuring Office 2004 programs, Virtual PC for Mac, and Messenger for Mac.
A new Maintenance section that describes the tasks and issues associated with updating and maintaining Office 2004 after deployment.
Updated and improved coverage of security issues.
A list of online resources that provide additional information helpful for deploying, administering, and using Office 2004.

We've also started planning the next release of the Resource Kit and other documentation. This is a good opportunity to provide feedback. You can take this open ended survey. The survey period ends on July 10th. You can also provide further information by sending e-mail to Mac IT Pro Feedback (macitfb@microsoft.com).

Lastly, we're looking for a few volunteers to help us improve our understanding of some approaches to Mac Office deployment and operations. If you’re interested, please see details on the survey page.

Rick

Currently playing in iTunes: Help Me by Van Morrison

The History of Mac Office, Part I

Rick Schaut — Sat, 03 Jun 2006 22:56:00 GMT

We've all seen the scene. A group of people are having a discussion about something, and one older, and allegedly wiser, person has been sitting back just letting the discussion ramble on, until they meet some sort of impasse, at which point everyone turns to this older, and allegedly wiser, person for the answer to some obscure question. A pregnant pause ensues while this older, and allegedly wiser, person ponders the situation. The pondering thought is, more often than not, something along the lines of, "How the hell did I get myself into this?"

Well, in a sort-of, kind-of, almost way, we've enacted this little scene out in the bloggosphere. Erik Schwiebert wrote about some past changes to the tool sets we've used to build various versions of Office, and closes his post with:

I don’t know what tools were used on versions of Mac Office prior to Office 98 (Rick Schaut would know about Word 5.1 and Word 6). MPW? Maybe, but that would have been a really long time ago! [Emphasis not added.]

Thanks, Schwieb! You've given me another item I can add to my "You know you're getting old when..." list.

Enough self-flagellation. Someone with the nom de plume of "odysseus" left this crumb at the end of Schwieb's post:

You haven’t mentioned anything about the fact that Mac Office versions all the way through Office 2001 were written in a mid-level interpreted language called “pcode.” Was that a consequence of using MSVC4?

Earlier versions of Microsoft's applications were compiled to a mid-level interpreted language. They weren't written in that language. They were still written in C with some hand-coded assembly where speed was crucial.

The term "P-Code" is based on the UCSD P-System, where the "P" stands for "Pseudo". In many ways, it was a precursor to the Java run-time and the .NET CLR. Code was compiled to run on a pseudo-machine, and you implemented this pseudo, or virtual, machine on the various systems where you wanted to run your code. We were supposed to get easy portability that way, but that was back in the time when operating systems basically gave applications a way to talk to the file system and nothing else. Since then, massive API differences have accrued to each of the various systems such that the CPU is no longer the primary issue when porting code from one system to another.

At this point, I'm going to have to digress a bit for the non-programmers who are reading this. The most important tool in a programmer's toolset isn't the compiler. It's not even the editor, despite the constant religious battles between proponents of the various editors in existence. Rather, the most important tool in a programmer's tool set is the debugger. That tool is the one that allows us to monitor the execution of the code we write. It's how we find where the code is wrong so we can fix it.

Now, there are different kinds of debuggers, but we can generally break them down into two categories: source-level and low-level. A low-level debugger only shows you the machine code that your compiler generated. A source-level debugger shows you the source code you wrote for the code that's currently being run by the computer.

The rest of the tale isn't quite so much about compilers. It's about debuggers. One of those debuggers will be familiar to old Mac hats: MacsBug. It was a low-level debugger. The other debuggers are more closely tied to the compilers we used. So, back to our tale.

Before we started using MSVC, we were using another internal compiler that ran on DOS (we're talking before Windows here). Folks who were around before 1990 also remember that it was very difficult to get a PC to talk to a Mac, so we used to have to connect our PCs to our Macs using SCSI cables. The P-Code debugger, which was a source-level debugger, ran on DOS, and talked to the Mac over the SCSI cable.

This had some unfortunate consequences. For example, you couldn't use the P-Code debugger and MacsBug at the same time. The P-Code debugger used a polling mechanism to determine if the P-Code interpreter was still running on the target Mac. Imagine a phone conversation. When you dropped into MacsBug, it was like the P-Code interpreter had lost the signal on the cell phone. The P-Code debugger would hang on for a short while waiting for the signal to come back, but it wouldn't take very long before the P-Code debugger would hang up the phone.

If you're starting to see visions of bear skins and stone knives, then your understanding of the situation isn't very far off the mark. But, it gets worse. For Mac Word 5.x, we implemented some of the features as plug-in code resources. These features included all of the proofing tools and the drawing tool in Word 5.x (known then as "Subdraw"). We built these plug-ins using Think C.

Think C had its own debugger, but that debugger didn't have a clue about P-Code. So, now we're using two different compilers, and three different debuggers. And, one of those debuggers is completely useless. Yuk! Throw loadable code resources into the mix, and you have an absolute mess.

One of the features of a debugger is the ability to set a breakpoint. A breakpoint is a location in the code where the debugger will pause the program kind of like hitting the pause button on a VCR or a DVD player. Once your program is paused, you can then examine the program's execution in more detail.

But loadable code resources are a problem. One of the reasons for using a code resource is the fact that it doesn't always have to be in memory. On a Mac that has only half a MB of RAM, being able to kick some code out of memory is a very useful feature. This feature, however, makes debugging a plug-in exceedingly difficult. You can't set a breakpoint in a code resource until it's loaded into memory.

So, let's summarize the situation. We have pieces of code built by two different compilers, neither of which understand the other. The debugger with Think C is completely useless, because it hasn't a clue about the main program that's running, which means the only way we could debug, say, the grammar checker was using MacsBug. But, every time we dropped into MacsBug, the P-Code debugger would whig out.

Those of you who've seen me in person now know that the thinning hair on the top of my head is not entirely due to male pattern baldness. By comparison, MSVC was heaven.

One last little tidbit: Back when we were using MSVC, someone had written two little programs, called victory.exe and defeat.exe. Each of them played two octaves of the C-Major scale through the PC's speaker. Victory.exe played the notes in ascending order, and defeat.exe played the notes in descending order. When your builds succeeded, the build system would run victory.exe, and when your builds failed, it would run defeat.exe. The cool part about that is you didn't have to keep watching your computer to see if the build had finished.

When we moved from MSVC to CodeWarrior, we lost this ability to play some custom sound. But, XCode has this feature. If you have XCode installed, type "defaults read com.apple.XCode | egrep Sound" in your terminal window to see if these are set. This sent me down a rather amusing path.

Because we're porting quite a bit of code from Win Office so we can support the new file formats, we have our own Win Office build machine set up in our lab (where we connect to the machine using Remote Desktop Connection). And, feeling a bit nostalgic, I decided to rummage around in their build system to see if they still had those sounds. Well, they have a couple of .wav files, but they're not the same sounds. In particular, the sound for a build failure is about a 15 second montage of Homer Simpson uttering his famous "Doh!" with various inflections.

I played this, and laughed a bit. Then it occurred to me that, as I had been setting this Win Office build machine up, the first couple of times I tried to build Win Office, the builds failed. I hadn't gotten the setup quite right yet.

I wonder if anyone was in the lab when one of those builds failed...

Rick

Currently playing in iTunes: Life Before Insanity by Gov't Mule

Mac BU's Lab on Your Mac Life

Rick Schaut — Thu, 25 May 2006 02:36:00 GMT

I just checked in on Your Mac Life to see what was in store for tonight's show, and it turns out that they'll be interviewing David Weiss about our MacLab.

I hope David remembers to remind Shawn about how to use page numbers in Word...

Rick

Currently playing in iTunes: The Low Spark Of High-Heeled Boys by Traffic

Flying Pigs

Rick Schaut — Thu, 06 Apr 2006 18:25:00 GMT

Last July, pigs flew through Hyde Park, and yesterday they flew through Cupertino.

The only thing I want to know is, of those who've installed Boot Camp, how many of them have dragged their task bar to the top of the screen?

Rick

Currently playing in iTunes: Land of Make Believe by Chuck Mangione

Of Intel Macs and Red Herrings

Rick Schaut — Sat, 25 Mar 2006 02:53:00 GMT

Scott Byer at Adobe put up a very nice post about the switch to intel and some of the growing pains many of us are going through. While I've added my own comments to Scott's piece, a number of my colleagues in Mac BU have asked me to weigh in on the subject here on my blog.

First off, it's difficult for me to discuss specific facts. I'm under NDA with Apple that precludes me from discussing specific issues with the transition, particularly with issues related to the tools.

Nevertheless, there are some red herrings out there, and I'm going to try to dispel a few of them.

1) Steve said it was just a recompile! What gives?

I've discussed this one before, but it bears repeating. When Steve Jobs did his demo at the last WWDC, he was talking to an audience of developers. He knew, as did every other developer in that room, that getting the code to recompile was only the first step in a long process of testing and verification. No two compilers generate code in exactly the same way, and no software developer worth the name goes through a compiler switch without extensive testing.

A few reports who've never written a line of code in their lives and who didn't actually take the time to go out and interview a variety of developers to get a different angle on the story propagated the notion that developers would be able to quickly move to Intel, but that's not what Steve was saying. Steve was only saying that the tools wouldn't be a major obstacle--or at least that Apple was prepared to get the tools to the point where they wouldn't be an obstacle.

2) Apple's been advising developers to move to XCode for years. All you had to do was follow Apple's advice.

At that time of Steve's speech at WWDC, most of the major applications weren't built using XCode. That's true for both Adobe and Microsoft. That fact alone complicates matters entirely. When this fact has been pointed out, this variation on the "Steve said it was just a recompile" meme has emerged. According to this meme, all we really needed to to was follow Apple's advice, and, well, we're the bad guys for not having done so.

While Apple did advise developers to move to XCode, Apple was rather tight-lipped as to the fundamental reason why. In the mean time, developers had to consider that advice along with the fact that Metrowerks' toolset was both significantly faster than the XCode/GCC combination and generated better code than XCode/GCC. No sane developer would sacrifice both a significant level of productivity and the quality of their product merely because Apple said so.

But all of that's very much beside the point. Whether we had gone through the pain of porting to XCode/GCC in some earlier release of our products, we'd have still had to go through this pain. The time spent doing this work then would have to have come from the features that we were, instead, adding to our programs. Arguing that we should have, somehow, absorbed this pain earlier really has little bearing on the nature and extent of the pain. We'd still have to do the work, and customers would have suffered as much then as they are now. The only difference is that, now, we have a tangibly legitimate reason for the suffering.

3) If you'd just ported from Carbon to Cocoa, your problems would have been solved.

I love this one. It so clearly demonstrates an astounding level of ignorance. I've already posted on the "Carbon vs Cocoa" issue, so I won't really belabor the point here. I'm just amazed to see people arguing that porting not only from one compiler to another but from one application framework to another would, somehow, magically be less expensive than simply porting from one compiler to the other.

4) Some apps have done it, why can't you?

In some ways, this is a legitimate criticism, but it glosses over significant differences between, say, PowerPoint or Photoshop and BBEdit. In this regard, there are two points worth considering.

The first is that the amount of work required to port a code base does not grow linearly in proportion to the size of the code base. There are a number of reasons for this, all of them related to complexity. For example, a more complex C++ program is far more likely to make use of certain language constructs (e.g. RTTI, templates and multiple inheritance) than would a less complex program. With more complex projects, the amount of work grows more on an exponential scale: i.e. twice the size of the code would require nearly ten times the amount of work to port.

The other is specifically related to GCC. GCC uses STABS to describe symbolic information so that the debugger can translate the code it's observing into symbols so that the programmer doesn't have to do this by hand. For those who care to take the time, RedHat has decent documentation on the STABS format.

The problem with STABS is that it's very verbose compared to other formats (particularly the xSYM format that Codewarrior uses). It's fairly easy to verify this. If you have XCode installed on your computer, create a simple "Hello World" Carbon project. Rename the main.c file to main.cpp (C++ generates mangled names due to overloading), and add a couple of simple classes to your app. Now, build both the debug configuration and the retail configuration. To be even safer, run strip on your retail build. Now, compare the difference in sizes between the two build.

I can't say which application, but I have it on good authority that a modern C++ application has actually hit the virtual memory wall (i.e. the combined code + generated symbols resulted in executable code larger than would fit in virtual memory). And, no, it's not an application that I've ever worked on or contributed to in any way.

Before I close this, I want to make two points very clear. First, I agree wholeheartedly with Scott's primary thesis: I can't imagine any developer who would prefer a long, drawn-out process of porting code from one build system to another to writing features that solve people's problems. We are already doing as much as we can possibly do. Indeed, one of the reasons I haven't posted very much as of late is because there really is a great deal of tedious work to do.

Secondly, I want to commend the tools group at Apple for the yeoman's job they've been doing to help us make the transition. I'm reluctant to name names, but I honestly wish I could. I've met with these people, and they really are an outstanding crew. Thanks, everybody. You know who you are.

Rick

Currently playing in iTunes: Dixie Chicken by Little Feat

A Ghost from Mac World Past

Rick Schaut — Thu, 19 Jan 2006 10:05:00 GMT

As many people at Mac World pointed out to me, I've been rather silent lately. There's a reason for that. Of the stuff that's on my mind lately, there just isn't a whole heck of a lot I can discuss publicly. Even telling you some of what I did at Mac World this year strays too closely to the realm of forbidden discourse.

So, I'll just relate a story from Mac World past. Right after we shipped Office 98, Ben Waldman decided to take us out to dinner at Fleur de Lys during that year's Mac World. Neat idea, except that I didn't get the e-mail telling us that this place is jacket-and-tie only until I was sitting in my hotel room at the ANA (now the Argent). Great! Now, what do I do?

Well, the dinner was Tuesday night, and I was scheduled to do booth duty Tuesday afternoon. Nordy's is just around the corner, and I was pretty certain I'd be able to get any alterations done by that evening, so I ducked in. Being pressed for time, I didn't pay close attention to prices. I was rather more concerned with whether or not the clothes actually fit.

There was a rack of items on sale, so I figured I'd be OK so long as I chose items from the rack. Everything there was, indeed, on sale, including the rather nice Harris tweed jacked I'd picked out, which turned out to be sale priced at $500.00 off.

Of course, I hadn't noticed that this jacket was sale priced at $500.00 off until we'd finished all the measurements, chalked all the lines and I was standing at the cash register with credit card in hand. At that point, it was either scratch the whole idea of dinner at Fleur du Lys or buy the jacket. So, I bought the jacket. It's still the most expensive item of clothing in my wardrobe.

There's always some kind of dinner thing at Mac World. This year, we had dinner, with some MVP's and members of the customer council, at Frisson, where the food was just as good as the food we'd had at Fleur du Lys, and it didn't cost me a tie and a jacket.

Rick

Update: I neglected to mention running into Dave at the Nisus booth. There was customer there who was asking about Nisus. He said he'd bought Word, even though he didn't want to, because Word had the features he needed. The customer then turned, looked at me, and saw the Word icon on my shirt, at which point Dave burst out laughing. I patted Dave on the back, asked him if he was OK, and offered to get him some water. Check out Dave's blog. He has some good comments on stuff found on the Mac World floor.

The Life from the album "Songs In A Minor" by Alicia Keys