Buggin' My Life Away : History

The History of Mac Office, Part I

Rick Schaut — Sat, 03 Jun 2006 22:56:00 GMT

We've all seen the scene. A group of people are having a discussion about something, and one older, and allegedly wiser, person has been sitting back just letting the discussion ramble on, until they meet some sort of impasse, at which point everyone turns to this older, and allegedly wiser, person for the answer to some obscure question. A pregnant pause ensues while this older, and allegedly wiser, person ponders the situation. The pondering thought is, more often than not, something along the lines of, "How the hell did I get myself into this?"

Well, in a sort-of, kind-of, almost way, we've enacted this little scene out in the bloggosphere. Erik Schwiebert wrote about some past changes to the tool sets we've used to build various versions of Office, and closes his post with:

I don’t know what tools were used on versions of Mac Office prior to Office 98 (Rick Schaut would know about Word 5.1 and Word 6). MPW? Maybe, but that would have been a really long time ago! [Emphasis not added.]

Thanks, Schwieb! You've given me another item I can add to my "You know you're getting old when..." list.

Enough self-flagellation. Someone with the nom de plume of "odysseus" left this crumb at the end of Schwieb's post:

You haven’t mentioned anything about the fact that Mac Office versions all the way through Office 2001 were written in a mid-level interpreted language called “pcode.” Was that a consequence of using MSVC4?

Earlier versions of Microsoft's applications were compiled to a mid-level interpreted language. They weren't written in that language. They were still written in C with some hand-coded assembly where speed was crucial.

The term "P-Code" is based on the UCSD P-System, where the "P" stands for "Pseudo". In many ways, it was a precursor to the Java run-time and the .NET CLR. Code was compiled to run on a pseudo-machine, and you implemented this pseudo, or virtual, machine on the various systems where you wanted to run your code. We were supposed to get easy portability that way, but that was back in the time when operating systems basically gave applications a way to talk to the file system and nothing else. Since then, massive API differences have accrued to each of the various systems such that the CPU is no longer the primary issue when porting code from one system to another.

At this point, I'm going to have to digress a bit for the non-programmers who are reading this. The most important tool in a programmer's toolset isn't the compiler. It's not even the editor, despite the constant religious battles between proponents of the various editors in existence. Rather, the most important tool in a programmer's tool set is the debugger. That tool is the one that allows us to monitor the execution of the code we write. It's how we find where the code is wrong so we can fix it.

Now, there are different kinds of debuggers, but we can generally break them down into two categories: source-level and low-level. A low-level debugger only shows you the machine code that your compiler generated. A source-level debugger shows you the source code you wrote for the code that's currently being run by the computer.

The rest of the tale isn't quite so much about compilers. It's about debuggers. One of those debuggers will be familiar to old Mac hats: MacsBug. It was a low-level debugger. The other debuggers are more closely tied to the compilers we used. So, back to our tale.

Before we started using MSVC, we were using another internal compiler that ran on DOS (we're talking before Windows here). Folks who were around before 1990 also remember that it was very difficult to get a PC to talk to a Mac, so we used to have to connect our PCs to our Macs using SCSI cables. The P-Code debugger, which was a source-level debugger, ran on DOS, and talked to the Mac over the SCSI cable.

This had some unfortunate consequences. For example, you couldn't use the P-Code debugger and MacsBug at the same time. The P-Code debugger used a polling mechanism to determine if the P-Code interpreter was still running on the target Mac. Imagine a phone conversation. When you dropped into MacsBug, it was like the P-Code interpreter had lost the signal on the cell phone. The P-Code debugger would hang on for a short while waiting for the signal to come back, but it wouldn't take very long before the P-Code debugger would hang up the phone.

If you're starting to see visions of bear skins and stone knives, then your understanding of the situation isn't very far off the mark. But, it gets worse. For Mac Word 5.x, we implemented some of the features as plug-in code resources. These features included all of the proofing tools and the drawing tool in Word 5.x (known then as "Subdraw"). We built these plug-ins using Think C.

Think C had its own debugger, but that debugger didn't have a clue about P-Code. So, now we're using two different compilers, and three different debuggers. And, one of those debuggers is completely useless. Yuk! Throw loadable code resources into the mix, and you have an absolute mess.

One of the features of a debugger is the ability to set a breakpoint. A breakpoint is a location in the code where the debugger will pause the program kind of like hitting the pause button on a VCR or a DVD player. Once your program is paused, you can then examine the program's execution in more detail.

But loadable code resources are a problem. One of the reasons for using a code resource is the fact that it doesn't always have to be in memory. On a Mac that has only half a MB of RAM, being able to kick some code out of memory is a very useful feature. This feature, however, makes debugging a plug-in exceedingly difficult. You can't set a breakpoint in a code resource until it's loaded into memory.

So, let's summarize the situation. We have pieces of code built by two different compilers, neither of which understand the other. The debugger with Think C is completely useless, because it hasn't a clue about the main program that's running, which means the only way we could debug, say, the grammar checker was using MacsBug. But, every time we dropped into MacsBug, the P-Code debugger would whig out.

Those of you who've seen me in person now know that the thinning hair on the top of my head is not entirely due to male pattern baldness. By comparison, MSVC was heaven.

One last little tidbit: Back when we were using MSVC, someone had written two little programs, called victory.exe and defeat.exe. Each of them played two octaves of the C-Major scale through the PC's speaker. Victory.exe played the notes in ascending order, and defeat.exe played the notes in descending order. When your builds succeeded, the build system would run victory.exe, and when your builds failed, it would run defeat.exe. The cool part about that is you didn't have to keep watching your computer to see if the build had finished.

When we moved from MSVC to CodeWarrior, we lost this ability to play some custom sound. But, XCode has this feature. If you have XCode installed, type "defaults read com.apple.XCode | egrep Sound" in your terminal window to see if these are set. This sent me down a rather amusing path.

Because we're porting quite a bit of code from Win Office so we can support the new file formats, we have our own Win Office build machine set up in our lab (where we connect to the machine using Remote Desktop Connection). And, feeling a bit nostalgic, I decided to rummage around in their build system to see if they still had those sounds. Well, they have a couple of .wav files, but they're not the same sounds. In particular, the sound for a build failure is about a 15 second montage of Homer Simpson uttering his famous "Doh!" with various inflections.

I played this, and laughed a bit. Then it occurred to me that, as I had been setting this Win Office build machine up, the first couple of times I tried to build Win Office, the builds failed. I hadn't gotten the setup quite right yet.

I wonder if anyone was in the lab when one of those builds failed...

Rick

Currently playing in iTunes: Life Before Insanity by Gov't Mule

A Ghost from Mac World Past

Rick Schaut — Thu, 19 Jan 2006 10:05:00 GMT

As many people at Mac World pointed out to me, I've been rather silent lately. There's a reason for that. Of the stuff that's on my mind lately, there just isn't a whole heck of a lot I can discuss publicly. Even telling you some of what I did at Mac World this year strays too closely to the realm of forbidden discourse.

So, I'll just relate a story from Mac World past. Right after we shipped Office 98, Ben Waldman decided to take us out to dinner at Fleur de Lys during that year's Mac World. Neat idea, except that I didn't get the e-mail telling us that this place is jacket-and-tie only until I was sitting in my hotel room at the ANA (now the Argent). Great! Now, what do I do?

Well, the dinner was Tuesday night, and I was scheduled to do booth duty Tuesday afternoon. Nordy's is just around the corner, and I was pretty certain I'd be able to get any alterations done by that evening, so I ducked in. Being pressed for time, I didn't pay close attention to prices. I was rather more concerned with whether or not the clothes actually fit.

There was a rack of items on sale, so I figured I'd be OK so long as I chose items from the rack. Everything there was, indeed, on sale, including the rather nice Harris tweed jacked I'd picked out, which turned out to be sale priced at $500.00 off.

Of course, I hadn't noticed that this jacket was sale priced at $500.00 off until we'd finished all the measurements, chalked all the lines and I was standing at the cash register with credit card in hand. At that point, it was either scratch the whole idea of dinner at Fleur du Lys or buy the jacket. So, I bought the jacket. It's still the most expensive item of clothing in my wardrobe.

There's always some kind of dinner thing at Mac World. This year, we had dinner, with some MVP's and members of the customer council, at Frisson, where the food was just as good as the food we'd had at Fleur du Lys, and it didn't cost me a tie and a jacket.

Rick

Update: I neglected to mention running into Dave at the Nisus booth. There was customer there who was asking about Nisus. He said he'd bought Word, even though he didn't want to, because Word had the features he needed. The customer then turned, looked at me, and saw the Word icon on my shirt, at which point Dave burst out laughing. I patted Dave on the back, asked him if he was OK, and offered to get him some water. Check out Dave's blog. He has some good comments on stuff found on the Mac World floor.

The Life from the album "Songs In A Minor" by Alicia Keys

Another Win Office Blogger

Rick Schaut — Wed, 16 Nov 2005 07:00:00 GMT

The list of Win Office 12 bloggers is growing. The latest addition is Will Kennedy, who is the general manager of the Outlook team.

Will's a former Word developer. Before that, Will was an intern on the Mac Word 5.0 project. Back in those days, we would occasionally "name" some of our testing releases--an effect carried out by changing the string that was used for new untitled documents in the debugging build. When you started up Word, the window title for a new, untitled document might, for example, be "FearAndLoathing1" (for the Fear and Loathing release) instead of "Document1".

When Will's internship was finished, we did a WillKennedy release, and, for a week, new, untitled documents would come up with "WillKennedy1" for the window title. As far as I know, Will's the only person to have had a testing release of Mac Word named after him.

Stop by, and say, "Hi" to Will. While you're at it, tell him how many customer accolades he'd receive if he went back to working on Mac Word.

Rick

Currently playing in iTunes: Wish You Were Here by Pink Floyd

Dave Luebbert on Channel 9

Rick Schaut — Sun, 13 Nov 2005 19:30:00 GMT

What was the Ray Gram Memorial Feature that shipped in Apple's System 7? Who invented the combo box (drop-down edit control)? What was the real downfall of WordPerfect? What was the project code name for Mac Word 5.0, and why?

You'll find answers to these questions and more in this Channel 9 interview of Dave Luebbert.

Dave Luebbert worked on Mac Word 1.0, Mac Word 3.0, Mac Word 4.0 and was the development lead on Mac Word 5.0.

Update: I just got some mail from Dave correcting my spelling of Ray's name (thanks Dave). Dave also pointed out that Ray attended the University of Wisconsin Milwaukee at nearly the same time I did.

Come to think of it, the last time I ran into Ray was at General Mitchell Airport in Milwaukee during some holiday season a few years back. One of these days, I'll have to look him up, and we can reminisce about stuff like the Safe House and Ma Fischer's.

Rick

Currently playing in iTunes: Famous Alto Break by Charlie Parker

Gettin' Published

Rick Schaut — Wed, 22 Jun 2005 17:57:00 GMT

Well, APress is advertising it, and Amazon is taking orders for it, so I suppose I can now publicly revel in the fact that Joel Spolsky selected my essay on Mac Word 6 for his collection of essays, The Best Software Writing I, Selected and Introduced by Joel Spolsky. Props go to Derek Miller at Pen Machine for nominating the essay in the first place.

Back in October, I'd noticed that Derek had nominated the essay for Joel's book. At the time, I felt honored, but didn't really think it would be selected. I was rather surprised when, a couple of months ago, Joel contacted me for permission to publish it.

I don't think of myself as a writer. Though people have told me that I should write more often, most of them have been members of my family or close friends. I'd mostly chalked that sort of thing up to unmitigated bias and moved on. Yet, almost simultaneously with Joel's request to publish my essay on Mac Word 6, I got a completely unrelated offer from a small publisher about writing a book.

Unfortunately, regardless of how my self-image might change because of this, it's still very hard to figure out when I'd actually have time to do some serious writing. For the foreseeable future, I'm going to be head-down getting Mac Word up to speed on the new XML-based file formats we just announced. After that, there's a lot of work to be done investigating universal binaries.

Who knows? Maybe I'll sit down and do some writing after Mac Office 12 ships. Still, I get the feeling that there's going to be some other issue with Mac Word that I won't want to let drop. After all, I've asked myself a variation of this same question following the completion of the previous 6 major releases of Mac Word, and I'm still doing this.

By the way, Joel, I do have a nice home, but it's in Woodinville not on Mercer Island; my mother is probably much more fond of the condo in Bothell than she would have been of a loft in Seattle; I rather doubt that anyone would mistake my Acura sedan for a Bently convertible; and we just sold the big boat.

Rick

Currently playing in iTunes: Savor/Toussaint L'Overture by Santana

Big Dave (Which One?)

Rick Schaut — Tue, 14 Jun 2005 07:36:00 GMT

Dave Winer provided this link to an article posted by Dave Luebbert. Dave Luebbert was the development manager for Mac Word 5.0, and was my first boss at Microsoft.

Dave (L) is an incredible guy. He's a musician, and a very big fan of Jazz. He's been working on a music composition program (Song Trellis) for as long as I can remember.

I learned a lot from Dave, not all of it related to developing software. You can, too, if you follow the link above. I highly recommend his article.

As for which "Dave" is "Big Dave"? Well, Dave L is no small guy. We had another Dave who worked on the Mac Word 5.0 project. He wasn't a little guy either, but Dave L still got the moniker "Bid Dave." I suppose one could say that Dave Winer is "big" in many ways, but Dave L will always be "Big Dave" to me.

Rick

Currently playing in iTunes: Famous Alto Break by Charlie Parker

MPW C Error Messages

Rick Schaut — Fri, 01 Apr 2005 13:54:00 GMT

Those of us who've been programming the Mac for a few years have some fond memories of the old Macintosh Programmer's Workbench (MPW). It was very much like the Terminal Window on OS X, but had a few quirks of its own.

Among those quirks were the error messages that the C compiler would give you for certain errors. Messages like, "a typedef name was a complete surprise to me at this point in your program," and "String literal too long (I let you have 512 characters, that's 3 more than ANSI said I should)" almost made you want to write broken code just to see what the compiler had to say.

The most complete list of the error messages I could find is here.

Well, rumor has it that there's a skunk works at Apple who are busy porting some of these error messages into XCode 2.0. And they say that Apple have lost their sense of humor...

Rick

Currently playing in iTunes: Won't Get Fooled Again by The Who

From Bravo to Word

Rick Schaut — Sun, 20 Jun 2004 09:03:00 GMT

From Bravo to Word

Not long after the Wired article about Word 5.1, someone sent me a link to this post on the Cult of Mac Blog. It mentions what I had thought to be a rather widely known fact: that most of the people who first worked on Word came from the Xerox PARC. In fact, the one person most responsible for most of the design ideas that permeate Word is Richard Brodie who also worked on Bravo while at the Xerox PARC.

Not surprisingly, a number of ideas that were first explored in Bravo found their way into Word. The Cult of Mac mentions the file format, but the statement that’s quoted isn’t quite accurate—at least insofar as it leaves out some important details.

The basic design goal behind Word’s file format was to be able to read in only that amount of information that was necessary to fill the document window with text. You can see the fruit of this today by conducting a little experiment:

Boot Word.
In a new document, type ‘=rand()’
Save this document as “SmallDoc”, and close it
Open a new untitled document
Type ‘=rand(100)’
Type <Cmd>-y about twenty times (until you have more than 100 pages of text).
Save this document as “BigDoc”, and close it.

You could, if you wanted to, grab a stopwatch and time the next few steps, which are:

Select “SmallDoc” from the “File” menu
Select “BigDoc” from the “File” menu

If you’re timing this, start the stopwatch when you mouse-up in each file name on the menu, and stop the stopwatch when you first see the insertion cursor blink.

The first thing you’ll note is that there is no appreciable difference between the amount of time it takes Word to open BigDoc and the amount of time it takes Word to open SmallDoc—this despite the huge difference in sizes. In my experiment, BigDoc is over 1 MB in size while SmallDoc is barely more than 24K. BigDoc is 40 times larger than SmallDoc, but I can’t tell the difference when I open the files.

Now, there are a few data structures that are stored in the file and are proportional to the amount of text in the file, but the actual data in them is so small that reading them in approaches constant time relative to the amount of time it takes to read in the actual text and formatting. The result is, even today, a file open time that is proportional to the size of your document window, not proportional to the size of your document.

The post on the Cult of Mac Blog quotes Bruce Damer’s claim that, “Bravo and BravoX stored out files by essentially just dumping the memory heap,” which is really a gross oversimplification. If the file format consisted of a straight dump of the memory heap, then opening a document would still take time proportional to the size of your document.

The formatting in a Word file, however, is allocated in blocks of 512 bytes. Formatting information is added to each of these blocks until they fill up, in which case new blocks are allocated. These blocks are written to the file as full 512-byte blocks whether they’re full or not, which is the only sense in which a Word file consists of a dump of the memory heap.

Damer attributes his claim to something Charles Simonyi said, but it’s almost certain that either Damer didn’t fully understand what Simonyi was saying or that Simonyi wasn’t entirely clear that this “memory dump” aspect of Word’s file format is limited to the disk pages that hold formatting information.

Rick

Anatomy of a Software Bug

Rick Schaut — Wed, 19 May 2004 20:50:00 GMT

Anatomy of a Software Bug

Chris Mason is the person who hired me to work at Microsoft. By the time he hired me, he’d already spent a great deal of time looking into the issue of general software quality, and had written a memo (known as the “Zero Defects” memo) that underlies much of our software practices today. The ideas have been refined since then, but they haven’t changed much in terms of the basic concepts.

One of my favorite Chris Mason quotes comes from that memo, “Since human beings themselves are not fully debugged yet, there will be bugs in your code no matter what you do.” We work to minimize the bugs in the software we ship, but they’ll always be there.

The problem stems from the overall complexity of the software. In this context, “complexity” doesn’t refer to the code itself. Rather, we’re talking about the shear volume of things the user can do. In Word, for example, we have:

More than 850 command functions (e.g. Bold and Italic are the same command function)
More than 1600 distinct commands (e.g. Bold and Italic are distinct commands)
At any given time roughly 50% of these commands are enabled (conservative estimate)
With just 3 steps, the possible combinations of code execution paths exceeds 500 million

Now, there’s a philosophical issue about the desirability of increasingly complex software, but I’m not going to discuss it here. For all practical purposes, I don’t think there’s much benefit to getting into a discussion about it. It may be an interesting question on some level, but it’s one we’ll never fully resolve. And I’m just not all that interested in getting bogged down in an endless debate without the possibility of resolution.

I mention the issue of complexity because it leads to subtle interactions that can be difficult to track down. To illustrate the point even further, I thought I’d discuss the anatomy of one of the more famous bugs we’ve had in Word: the “Disk is full” on save error. Before I do, however, I should point out that Pierre Igot, after some prodding on my part, did provide us with a sample document that helped us to track down one of the more subtle interactions involved in this particular problem. For that, Pierre, I thank you and so do Word users everywhere.

The story of this problem begins with a basic design decision made when Richard Brodie was Word’s primary software architect. Brodie came to Microsoft along with Charles Simonyi after working at the Xerox PARC where he’d worked on Bravo—their version of the GUI word processor. A number of the ideas used in Word came from that early effort. Brodie joined Microsoft in 1981, began work on Word in the summer of 1982, and finished version 1.0 in October of 1983. You can read about much of the story in Microsoft First Generation by Cheryl Tsang.

Brodie figured out that a document is really just a collection of pieces of text, and that it didn’t really matter where each piece of text is physically located within the document’s file. For that matter, you could have one piece of text that came from one file and another piece of text that came from another file. We refer to this collection of pieces of text as the “piece table.” This design has a number of benefits. For example, if you copy text from one document to another, you don’t have to actually copy the text from one file to another—at least not right away. All you really need to do is copy the appropriate entries in the piece table in the source document to the piece table in the destination document. Of course, you do need to copy the physical text and formatting from one file to the other when you save the destination document, but delaying that physical copy until save time meant that the actual copy/paste could be done very quickly.

This design also made implementing undo rather simple. In fact, according to Brodie, implementing undo was the primary reason to use this design. With this design, all you have to do is create an internal undo document. When the user deletes some text from the current document, for example, you copy the deleted entries from the piece table to the undo document and save some information about where those piece-table entries had been located in the original document. To undo the delete, you just copy the piece-table entries from the undo document back to the original document.

This design does have one problem: where do you put the text that the user types into the document if it doesn’t go into the file that’s behind the document? To solve that, Brodie added something called a “scratch” file, and the scratch file remains a core part of Word’s design to this day. On the Mac, Word creates this file in your TemporaryItems folder. On Mac OS X, this folder is located at /private/tmp/<UID>/TemporaryItems, where “UID” is your user ID number (for most people, that’s 501, but it can be a different number altogether depending on how your user account was created). If start up Word, open the terminal window and get a listing of your TemporaryItems folder, you’ll see a file named something like “Word Work File S_.” There may be a number after the “_” character. That’s Word’s scratch file (the “S” standing for scratch). You might also see one or more files named “Word Work File D_” with some number after the “_” character. This is a back-up copy of a document file (the “D” standing for “Document”).

At this point, we need to fast-forward the story by a decade to the next major feature that brought this problem to the fore: multiple undo. For ten years, Word had undo, but it was just a single-level undo. For Word 6, we added the ability to go back and undo every change you’d made to the document since you first started editing it. And, with Word’s document/file architecture, this wasn’t all that difficult to do: just make the undo document contain multiple records with one record for each change to the main document. It’s a very cool feature, and most of us couldn’t think of how we’d survive without it. But it leads to a problem.

It’s not uncommon for users to make a few edits to a document, save the document, make a few more edits, save the document again, make a few more changes, and continue this process of edit/save for hours on end. Each time you delete text, however, the actual text itself exists in the last-saved file for the document you’re editing, and, with multiple levels of undo, the undo records for text deletions still point back to the last-saved version of the document’s file before you deleted the text. The next time you save, Word can’t close the last-saved version of the file, because the undo document still contains a reference to it. So, if you keep editing and saving, you’ll eventually hit an open file limit. At least this was true of Word 6. It’s changed quite a bit since then.

Arguably, this is something we should have figured out before we shipped Word 6, but, as Chris Mason pointed out, we humans haven’t been fully debugged yet. Moreover, it’s easy to say that one should have thought of a particular interaction in a complex piece of software, but that’s way easier said than done. When you’re implementing any given feature, you’re totally focused on the basic problems involved in the feature itself. To put this into perspective, the person who implemented multiple undo in Word is one of the best developers who has ever worked on Word, and has, since, been recognized as a Microsoft Distinguished Engineer.

The reality is, that we hadn’t realized we’d created this situation when we added multiple levels of undo. Moreover, this problem has several different variations on the basic theme. At this point, the story involves our efforts to understand the nature and scope of the problem, and to come up with the “best” way to fix it. Because of the variations, however, the problem has been like an onion. We’d peel away one layer of the onion, only to find some other variation that we hadn’t, for various reasons, figured out before.

As we weave our way through the rest of the story, there are some important points to keep in mind. The first is that I can’t fix what I can’t see, and, where software bugs are concerned, “seeing” means being able to watch the program execute, via some debugging tool, at the key point in the execution of the code where the problem occurs. In order to do this, I have to have a precise set of steps that consistently reproduces the problem. This not all that different from the problem a mechanic faces when trying to figure out the cause of that mysterious engine noise that only occurs after you’ve been driving the car around town for a few hours.

The second is that this particular problem is a developer’s worst nightmare. The fundamental cause is a basic design decision that you made more than a decade ago, and the only way to really fix it for certain is to rewrite the entire application from the ground up. Since that’s simply not an option for a product that you’ve shipped several times, you’re left with trying to make the problem difficult for most users to run into while trying to also minimize the negative effects if the user should ever run into the problem. This approach can, unfortunately, lead you to believe that you’ve come up with an “optimal” fix only to discover later that there’s another facet you haven’t taken into consideration (because you didn’t even know it existed until you peeled away the previous layer of the onion).

The third point to keep in mind is that we in Mac BU have relatively limited resources. When there’s a problem that’s fundamental to Word itself, we tend to let our Win Word siblings focus on that problem. Our efforts tend to have little chance of adding to their efforts, and this frees us up to focus on problems specific to Mac users. In general, this is the most efficient way to handle problems that our users are having, but there can be instances where there’s a Mac-specific dimension to a problem. As we’ll see soon enough, this particular problem had a Mac-specific dimension that complicated our efforts to fix it, and it took us a while to find that Mac-specific dimension.

Lastly, the fact that Mac Word’s code base has been forked from Win Word’s means that the Win Word people can make a change in the code for one reason, and that change can have other side-effects that we won’t see in the Mac version until we run into some very specific circumstances that show us the different behaviors caused by this change. In this particular case, Win Word added two lines of code in a routine that would seemingly be completely unrelated to this problem, but also made this problem much more difficult for users to run into in Win Word than it was in Mac Word. This one is the last piece (maybe I should say latest piece) in the puzzle that we discovered only a few months ago.

Whew! That’s a lot to keep in the back of our heads, but, nonetheless, let’s rewind back about ten years. Word 6.0 has just shipped on Windows, and we’re pretty happy with people’s reactions to the product. It doesn’t take long, though, for us to figure out that there’s a fly in the ointment. Reports start trickling in about people editing their documents “for a while” at which point they try to save their document and they get a “Disk is full” error. We’d ask people what they were doing, and the response was always some form of vague notion that they’d just been editing their document “for a while.” The precise measurement of “for a while” varied from user to user. For some folks, it was a little over an hour. For others, it was several hours. Reproducing the problem appeared to be highly dependant upon the user’s work habits.

After several months of trying to figure out the problem, someone in testing wrote a macro that inserted a large amount of text into the document and then, in a loop, replace successive words within the document saving it after each replace. Run this macro for a while, and you get a “Disk is full” error on one of the saves, at which point you can no longer save your document. Cool! We now have steps that reproduce the problem.

So, this document got handed off to a developer, who then fired up Word under the debugger, opened the document and ran the macro. The problem “reproduced,” but, for reasons that weren’t apparent at the time, the error that the developer ran into was subtly different from the error that the tester ran into. The developer thought about the problem he was seeing, and came up with one of those “optimal” fixes I mentioned above. It was the “right” fix in terms of the problem the developer saw, but it wasn’t the “right” fix for the problem that the tester saw.

What was this subtle difference between what the developer saw and what the tester saw? As I mentioned above, the basic theme of the problem is to hit an open file limit. In this case, there are two limits: Word’s internal open file limit and the OS’ open file limit. It turns out that the debugger bumps the OS’ open file limit from what it would normally be when you run Word outside the debugger. When the tester ran the macro, Word hit the OS’ open file limit. When the developer ran the macro, with Word running under the debugger, Word ran into its own, internal, file limit.

After a few iterations of the tester saying, “Sorry, but the bug’s not fixed yet,” and the developer saying, “What are you talking about? I don’t see the problem!” they both figured out that they were seeing different errors. Crap! The problem only reproduces when you’re not running under the debugger, which removes the one case where the developer can actually see what’s going on. At this point, we have yet to figure out that the problem involves hitting the OS’ open file limit. At this point, though, the developer isn’t completely in the dark, and comes up with a fix for the tester’s problem.

As I pointed out, the problem involves the undo document having a reference to the previously saved-version of the document’s file. The developer’s original fix was to add some code, in the case where Word hit its internal open file limit, that would basically remove everything from the undo document (what we refer to as “nuking the undo stack”). Nuking the undo stack allows the save to proceed, because Word can now close the open files that were referenced by the undo document. However, since the tester was seeing a different error, the developer’s fix didn’t handle that case.

Nonetheless, the developer took a different approach. Knowing that the undo document was very likely to be involved, one could walk through the undo document, and copy the text for any pieces that pointed to the previously-saved version of the document’s file to the scratch file. He coded up the solution, and handed a buddy-build off to the tester. The tester ran the macro, and the problem was fixed. The first layer of the onion had been peeled away, but the fix still wasn’t an “optimal” fix. As it stood, the chances that a user would run into the problem had been greatly reduced, but we still hadn’t dealt with the “minimize the damage if they do hit it” side of the issue. That’s because, at this point, we had yet to understand that the problem outside the debugger had to do with the OS’ open file limit. Because this problem wouldn’t reproduce under the debugger, the developer had no way of knowing exactly where the failure was occurring. Without knowing that, the developer didn’t know where to add the code that would “nuke the undo stack.”

To give you a sense of the time frame, this fix was ported from Win Word to Mac Word during the Office 2001 development cycle, and was back-ported into Word 98 for a service release that was done not too long after that. It’s also at this point where the Win Word and Mac Word stories diverge. There are two reasons for this. The first is that this was the point in time where Win Word got that two-line code change that I mentioned above. The second is that the open file limit under Mac OS is different than it is under Windows. I might be mistaken on this point, but I think the open file limit under Mac OS X is different from the limit under Mac OS 9 as well.

At this point, we still didn’t know that the basic problem involved hitting the OS’ open file limit. After a while, though, we did know that Mac Word users were seeing this problem way more often than Win Word users. In fact, the difference was enough for Mac Word testers to start investigating the problem directly. One of the things we did know is that the problem involved file references in the undo document. So, we came up with a variation of the original fix.

In order to understand this, we have to understand a basic principle of fixes. You make the simplest code change required to fix the problem. This reduces the chances that the fix will cause some other problem that is, potentially, worse than the one you’re trying to fix. When you’re mucking about with the locations where data is stored in files, the potential for catastrophic problems resulting from your fix is high. In that sense, the original fix for this problem was limited to copying what might be known as “simple” pieces. A “simple” piece has only text. A “complex” piece might have a graphic, or it might involve a field in the document, both of which are likely to have data in the file in addition to the text itself.

With this in mind, for Mac Word X, we modified the notion of what would be a “simple” piece of text for the sake of deciding whether or not to copy a piece from the previously-saved document’s file over to the scratch file. To view this in a slightly different way, we made the code that copies undo document referents more aggressive. This resolved another test case that the Mac Word testers had developed, again using a slightly different macro that would eventually cause the “Disk is full” error to occur. This fix didn’t actually make it into the shipping release of Mac Office X, but it was included in a subsequent SR (I don’t recall specifically which one).

At this point, we still don’t know that the problem involves the OS’ open file limit. That discovery didn’t happen until this past summer when, through the very persistent efforts of Mac Word’s current lead tester, we were able to use some tools on Mac OS X to figure out exactly what was happening. While we were able to verify this, we still didn’t know the exact location where Word was failing to open a file due to having hit the OS’ open file limit. Again, we still can’t get this to reproduce under the debugger, and there are a couple of places in the save process where it can fail because the OS won’t let Word open the file. So, rather than scatter fixes all over the place, we went with the sure fix: lower Word’s internal open file limit so we hit it before we hit the OS’ open file limit. This allows the code that nukes the undo stack to kick in, and then save the succeeds.

This brings us to late February/early March of this year, and the discussion I’d had with Pierre. While we still can’t reproduce the actual file open failure under the debugger, we now have enough information about what causes the problem to be able to predict when the failure will eventually occur. From that, we knew enough about the bug for me to believe that Pierre shouldn’t still be hitting that “Disk is full” save error in the version of Word he was using. Yet, he was still running into the problem.

That was the bad news: there was something about this problem that we still didn’t fully understand. However, armed with a sample document and the ability to predict when the error will occur, I could do something we’d never been able to do before: set up Word under the debugger, perform some steps in Word to see if those steps caused the predictive condition to occur, and set breakpoints that would tell me exactly why we weren’t able to copy pieces from the undo document over to the scratch file.

This is where I discovered those two lines of code that had been added to Win Word so long ago, yet hadn’t been added to Mac Word. When Word lays out a page in page layout view and a header or footer is visible, it updates any fields in the header or footer. If you have, say, a page field in the visible header/footer, Word will update that field. This is particularly necessary when you have the footer of one page and the header of the following page both visible in the document window. Word has to layout two pages in the same update, so it updates fields for the first page footer, lays out that page, then updates the fields for the next page header and lays out that page.

Now, why would this result in a field being copied over to the undo document? Well, Word has something called “auto undo tracking.” Basically, when you’re typing, Word automatically tracks the changes you’re making until you do something that causes Word to close out the “typing” undo record. You can see this when you click on the “Undo” dropdown on the standard toolbar. You’ll see “Typing <text you typed>” at various locations in the dropdown interspersed with other actions you’ve taken.

The two lines of code that were added to Win Word paused automatic undo tracking while updating these fields in the header or footer during page layout, then un-paused automatic undo tracking once the field update was finished. Ugh! How, on earth, were we to ever figure out that these two lines of code were the primary reason Win Word users weren’t seeing this problem nearly as often as Mac Word users? In any event, if you’ve stayed with me long enough, here’s a tip you can use until we release an SR of Word X (or earlier) with this fix. If you have a document that has headers and footers with page fields in them, do your editing in Normal view, and you’ll likely never hit the “Disk is full” save error.

Right about now, you’re probably asking, “Why did it take so long to figure out what was up with this?” Well, you might as well ask why police departments continue to have a large number of unsolved crimes on the books. The issue is the same: the investigation stalls for the lack of any further leads to follow. For the same reason that the police can’t just go out and start arresting anyone who might be a suspect, we can’t go scattering potential fixes throughout the code. Until we figure out what the precise nature of the problem is, we need leads that we can follow. The mere fact that you’re running into a particular problem isn’t a lead that I can follow. Specific details about potential suspects, however, are leads I can follow. When it comes to software problems, leads I can follow consist of information that helps me to reproduce the problems consistently.

And, always remember that I can’t fix what I can’t see. I have to be able to reproduce the problem while being able to run some kind of diagnostic tool. The key to fixing a bug is predictability. Without predictability, I can’t fix it, because without predictability I have no way to understand how the complex interactions in modern software cause the specific problem to occur.

Rick

Do You Remember?

Rick Schaut — Fri, 14 May 2004 17:53:00 GMT

Do You Remember

So, I blogged about life in Mac BU, and Scoble picked up on it. It’s always interesting when this happens, because some of the comments from Scoble’s readers can be a bit amusing. For example, Robert noted Microsoft’s early investment in the Macintosh platform, and one of his readers questioned when that investment ever happened. I don’t know. Maybe he thinks that creating software doesn’t cost anything.

Anyway, it got me thinking about that original software investment and how it all started, and I thought, “Hm… I wonder if anyone remembers the Software Dating Game.” For those who don’t know or don’t remember, the first Apple sales meeting for the Macintosh featured a spoof on the old Dating Game with Steve Jobs asking questions on behalf of the new lady, Macintosh, of three potential suitors in the form of executives of different software companies. The problem is, I could only remember Bill Gates and Mitch Kapor as two of the suitors, but I couldn’t remember the third (or even the company he represented). So, I did a little googling, and ran into this—gads, do they look young! Ah, yes. Fred Gibbons. Software Publishing Corp. Harvard Graphics. How could I forget?

By the way, the ToastDesign site has quite a few items to facilitate a nice little trip down memory lane. I like this one. Check out the two pictures on the top of the right-most column. Remember the “Portable” Mac? I think about that, and then think about the 15” PowerBook G4 that’s sitting on my lap as I type this, and I can’t help but marvel at the operation of Moore’s Law.

Of course, some of the memories are not so pleasant: remote debugging using a PC connected to the Mac via SCSI interface (shudder). But, it’s still good to remember how far you’ve come even if there were a few folks who didn’t quite figure out how to come along for the ride.

Rick

Where's Rick?

Rick Schaut — Sat, 01 May 2004 01:57:00 GMT

Where's Rick?

Wow, leave blogging aside for a while, and the world just passes you by. The past few weeks have been so busy, that I haven’t even been able to read the blogs I follow. I fired up NetNewsWire this evening, and there were over 700 posts awating my perusal. Clearly, I’m not going to get through them all.

So, why have I not been blogging lately? Well, we sent Mac Office 2004 off to manufacturing, at which point we all took a few days to just decompress. At the end of a product cycle, you tend to not even want to look at a computer, let alone sit down in front of one in order to type out a few random thoughts. Unfortunately, my boat was out of the water getting a new coat of paint on the bottom, so I didn’t get to do much sailing either. Yes, my timing was impeccable.

Also, the end of April is a very busy time for members of the Bahá'í Faith. We have three holy days, an annual meeting, our regular monthly meeting (monthly in terms of the Bahá'í calendar, not the Gregorian calendar—don’t ask), and a national convention all packed into the span just short of two weeks. In any event, my evenings have been pretty clogged, and, in fact, I have another meeting to go to tonight.

After the decompression time was over, it was back to work where I’ve had three major items on my plate. The first is engineering excellence planning which is where we take a look at some of our development methodologies and think about ways we can improve the process. As part of this, I’ve been working on writing some new unit testing code for Word which my Win Word counterparts are interested in stealing when I’m done (and I’m almost there).

The second is some product planning for Office 12. I’ve blogged about this in broad strokes before. Basically, we take some prototypical users, and we design some product use scenarios around those users’ needs. I’m a member of a cross-discipline team that’s drilling down into one of those scenarios. There is a different team drilling down into each scenario, understandiing what users are doing, looking into how our products presently help them solve the problems they’re facing and identifying what might be referred to as users’ “pain points” as they try to solve these higher-level problems. From this work, we’ll design the features that will go into the next version of Mac Office.

The third big item on my plate was the Customer Council meeting. The Customer Council consists of a number of customers who have purchased premium support. One of the things you can have with premium support is the ability to sit down with the product teams and discuss what your issues are. We had representatives from firms like the Rand Corporation, Turner Broadcasting, Gannett News, and several others. I think there were about 25-30 representatives present.

Normally, presentations at the Customer Council meetings are handled by program managers, but, following the release of Office 2004 to manufacturing, we had some personnel changes in the program management group affecting Word. So, I got to be the one making the presentation for Word, which is something I really enjoy doing and don’t often get to do because I’m a developer. The other cool thing about it was, after I’d finished my presentation and discussed some of the future directions we plan on taking with the next version of Office, none of the members of the Customer Council had any further questions. Members of the Customer Council don’t pull their punches, so when they don’t have any further questions or comments after you’ve finished the presentation, you’ve pretty much nailed it.

The other very cool thing about this year’s Customer Council is that we spent a good chunk of time sitting down with representatives from Apple, and the level of communication in the room was appreciably better than anything I’ve seen in the past almost 14 years. The results were so positive, that we think we want to do this kind of thing, sit down with customers, representatives from Apple and representatives Microsoft all at the same time to hammer out some issues. I don’t think it will be all that long before users reap some benefits from this kind of cross-communication. It’s very much a sea-change, and, I think, a very welcome development for the industry.

Anyway, so while I’ve been doing all this, Chris Pratley has been on paternity leave (congratulations on the baby, Chris). He’s been blogging, and blogging, and blogging and blogging about some of the history of Word. The posts provide a nice compliment to my post on Mac Word 6.0. Unfortunately, in the process, Chris discussed some of the things I had in mind.

One of the issues I wanted to talk about is why there is no “reveal codes” feature in Word. People ask me about this relatively often, and, when they do, I know I’m talking to former Word Perfect users. As Chris points out, Word Perfect used markup codes to do things like turn on bold, then, later, turn it off, similar to the way HTML works, while Word uses what might be described as a parallel array of runs of text matched to buckets of formatting properties (well, actually, they’re coded as differences from the underlying style, but that’s a detail for another post). One of the things Chris doesn’t point out, however, is how Word’s design is better suited for WYSIWYG word processors.

Suppose I give you an arbitrary location in the document (because, say, that happens to be the first character of the first line being displayed in the current document window), and ask you whether or not the character at that location is bold. Under the markup design, you have to start from the beginning of the document, and scan forward to the arbitrary location I’ve given you. Under Word’s design, you merely search for the corresponding run of text that contains that document location, and go look at the property bucket to see if bold is there. In Computer Science terminology, Word Perfect’s design requires an O(n) search, while Word’s design requires an O(log(n)) search.

Now, this didn’t mean all that much back in the days when the word processor presented the document on a character-based display. In fact, it actually put Word at a bit of a handicap, because Word was doing extra work to maintain the parallel buckets for all the formatting informtion. Word Perfect could just dump whatever characters you typed into the file without having to worry about ensuring that any formatting that’s applied to the newly-typed text gets stored as well as the new text. When people start using Windows, however, Word’s design had the advantage while Word Perfect’s design was at a disadvantage. This is yet another data point showing why we need to design the UI first, and then implement the features. It’s also something to keep in mind when people think about the history of word processors.

Rick

BruceO's Prank

Rick Schaut — Sun, 07 Mar 2004 08:13:00 GMT

BruceO's Prank

Microsoft is a great place to work, even if you are a Mad Mac Maven. There are a lot of reasons for that, but the best is that you get to work with some pretty cool people. I’ve already mentioned Ed Fries (which is pronounced “Freeze,” by the way). Another is Bruce Oberg.

Bruce is currently the head of Sucker Punch Productions, a software company that writes games for Nintendo and Sony PlayStation. Before starting up Sucker Punch, he was my boss on Mac Word 98. Bruce’s Purple Team is the only two-time winner of The Game. Bruce was the Special Guest Host at the infamous MacHack ‘94, where the Wednesday night opening session with Andy Hertzfeld extended well into the wee hours of Thursday morning aided by 75 pizzas delivered from Dominos. Bruce’s vanity license plate reads “OTTFFSS.” I guess you could say that he makes it count.

Bruce also orchestrated one of the best pranks ever pulled off at Microsoft.

A little background is necessary. Every business unit at Microsoft has at least one person who acts as the administrative glue that holds the whole unit together. At one time, they were called Group Assistants, but they’re now known as Administrative Assistants or Admins. They are indispensable to Microsoft’s overall functioning. If you really want to deal Microsoft a blow, hire away all the admins.

Admins report to specific unit managers. When managers move around, the admins go with them. So, when Ed Fries left the Word group, he took Pam Wagner with him. Pam was greatly loved in the Word group. She was friendly, efficient, and was good at doing things like finding snacks for 30 developers who work pulling a late-night Workaholic Wednesday. I still remember her spending months returning all the “Ship-it” awards to people in buildings 16, 17 and 18 after a group of Word developers had collected them all and stacked them in Ed’s office. Not once did she complain about the task.

Ed’s replacement, Antoine Leblond, had the unenviable task of hiring a replacement for Pam, and it was taking for ever. Dev leads were filing their own expense reports, we were bartering for the equipment we needed, and the storeroom was packed to the gills with stuff that wasn’t being offloaded to some other group that might need our surplus. Things were getting desperate.

On Friday, March 29^th, 1996, Antoine announced that he’d found a new group assistant. Her name was Meredyth, and she’d be starting the following Monday. At last, relief was in sight.

By mid-morning Monday, it was pretty clear that Meredyth was no Pam. Part of that had to do with the incense she burned in her office, the affirmations she put up on the wall (“I deserve to succeed! Yes, I do!”) and the bright pink name-tag on the door along with the note she put beneath it inviting everyone to, “Please stop in and say hi.” Another had to do with the heart-shaped stickers she put on everyone’s door—well everyone except Mark Walker who got one on his shirt.

By early afternoon, things were starting to really look amiss. Meredyth went to everyone’s office to schedule a fire drill. Notification of that sort of thing came from facilities, and you received it via e-mail. Later, Pam showed up to help Meredyth get acclimated, and that resulted in a number of indignant outbursts from Meredyth directed at Pam. The latter was enough to have several Word developers ready to show Meredyth the door regardless of what Antoine wanted. Whatever way Pam did anything, large or small, Meredyth did it in exactly the opposite way.

By the weekly 3:00 pm team meeting, mutiny was in the air. Antoine introduced Meredyth, at which point Meredyth delivered a teary tale about people not being nice to her saying that while this was her first day at Microsoft, it would also be her last. The outburst ended when she stormed out of the room. A minute latter, she came back into the room still teary-eyed, and said, ‘I forgot my purse.” She picked up her purse, headed back to the door, and, just before exiting, said, “And, by the way, April Fools!”

A minute later, she came back into the room to thunderous applause. “Meredyth” was a Seattle actress whom Bruce had hired for the day. At the time, she worked for Anything Entertainment, but I don’t know if she still does.

On Tuesday, we got e-mail from facilities saying that we’d be having a fire inspection and alarm test on Thursday.

You can read Bruce’s account here.

Rick

Mac Word 6.0

Rick Schaut — Thu, 26 Feb 2004 10:08:00 GMT

Mom always said, “The only good thing about beating your head against the wall is that it feels good when you stop.” Well, sorry Mom, but that’s not fully true. While you’re sitting on the couch buried beneath an ice pack, you tend to come up with a few ways to mind your head.

Shipping a crappy product is a lot like beating your head against the wall. It really does feel good when you ship a great product as a follow-up, and it really does motivate you to spend some time trying to figure out how not to ship a crappy product again.

Mac Word 6.0 was a crappy product. And, we spent some time trying to figure out how not to do that again. In the process, we learned a few things, not the least of which was the meaning of the term “Mac-like.”

In order to understand why Mac Word 6.0 was a crappy product, we need to understand both the historical background that led to some key decisions, and we need to understand some of the technical problems that resulted from those decisions.

Mac Word 5 and Pyramid

On October 5, 1991, we shipped Mac Word 5.0. The reviews were glowing. For the effort, we received the Mac software equivalent of a Tony award: the Mac World Eddy. Even today, there are people who say that Mac Word 5.0/5.1 comprise the best version of Mac Word we’ve ever shipped.

While Mac Word 5 was a great product, there was one problem with it: Win Word 2. They both shipped at about the same time, but Win Word 2 had more features (most notably a macro language, but there were a few others). This was a major sore point for Mac Word users. They wanted feature parity, and they wanted it now! The longer they had to wait for feature parity between Win Word and Mac Word, the more we got raked over the coals.

But we had a problem. Actually, we had a couple problems, the first being that Win Word and Mac Word were built from separate code bases. The other problem was Word Perfect. At that time, it still represented a major competitor on Windows, and we still had some catch-up work to do in order to get better than Word Perfect. If we had continued to develop Mac Word and Win Word from separate code bases, Mac Word would never have caught up to Win Word in terms of feature parity.

As of October of 1991, we already had a plan to address the first problem: the Pyramid project. It was a complete rewrite of Word intended both to address some nagging issues with what had, by that time, become somewhat of a crusty code base and to address the separate code base problem. Both Win Word and Mac Word would be built from that same code base.

Exit Jeff Raikes, Enter Chris Peters

Feature parity problem solved. Well, not quite. At the same time, Jeff Raikes was promoted from Word business unit manager to some other position in Microsoft (I forget exactly which), and Chris Peters was promoted to fill Jeff Raikes’ position. Most everyone knows about Jeff Raikes these days. Chris Peters, however, had been the development manager for Excel before moving to Word. His favorite pastime is bowling, and he was known for having huge stacks of empty Coke cans in his office.

While Jeff Raikes thought the Pyramid project as a good idea, Chris Peters looked at the Word Perfect problem and decided that Pyramid was a bad way to solve the feature parity problem. A complete code rewrite is risky. The whole point of a complete rewrite is to take a few steps backward in the short-run in order to be able to make some greater strides in the long-run. Chris Peters decided that we couldn’t afford to take the short-run hit that Pyramid required.

So, Chris Peters killed Pyramid. At that point, the only way to solve the feature parity issue is to start both Mac Word and Win Word from the Win Word 2.0 code base, which is exactly what we did.

But, that’s not the full effect of Chris Peters’ decision. At the time Chris Mason was the development manager for Word, and he strongly disagreed with Chris Peters’ decision. As a result, Chris Mason left the Word group to work on other things at Microsoft. Chris Mason understood the Mac, and had been a Word developer going back to Mac Word 3.0. Chris Mason was replaced by Ed Fries, who was far less of a Mac person than Chris Mason was, so we lost a good bit of Mac understanding in the higher-level management of the Word group. While it’s impossible to say exactly what effect this had, there’s a high probability that some of the trade-offs we made with Mac Word 6.0 would have gone in a different direction.

Technical Hurdles

Starting from the Win Word 2.0 code base presented a couple of technical problems for those of us on the Mac side. The first was that it was written to the Windows APIs. Solving this problem isn’t simply a matter of writing a layer that emulates the Windows APIs on the Mac. The way the two systems handle windows are fundamentally different, though it’s interesting to note that the new Carbon APIs are far more similar to the way Windows does things. The biggest problem is that Windows has the concept of child windows, while the Mac does not. The other is that, on Windows, everything is a subclass of the Window object. Even controls are Windows.

The other problem was a limitation in the Mac OS. While 68K Classic Mac OS was a nice operating system, it had one very glaring flaw. It didn’t do memory management very well. In fact, it barely did any memory management at all. Users had to tell the OS how much memory a program needed in order to run, and that’s how much memory the program got regardless of what the program might need at any given time during execution.

The memory problem was worse on 68K machines, because the memory given to a program, regardless of the virtual memory settings, was what the program got to use for both code and data. Under 68K, code was contained in something called a “Code Resource”. Now, you could swap these code resources in and out of memory as needed, which meant that the actual memory needs of your program could change drastically depending on what the user wanted to do.

For example, consider a grammar checker. The user isn’t going to want to check grammar all the time, so the grammar checker doesn’t need to be loaded into memory all the time. But a grammar checker isn’t a simple piece of code. It’s a memory pig. The way 68K Classic Mac OS handled memory meant that you had to set a minimum amount of memory for your application such that you could load that memory pig of a grammar checker.

I’m making a distinction between 68K Classic Mac OS and PowerPC Classic Mac OS, because Apple changed how code was stored, loaded and executed on the PowerPC. For those of you who remember, when you did Get Info on an application, it would show you two different memory requirements: one with virtual memory turned on and one with virtual memory turned off. With virtual memory turned on, the application’s code could be handled through something called “demand paged” virtual memory, so the code no longer had to fit in the application’s memory partition. That notorious grammar checker didn’t have to be given account when trying to figure out the application’s minimum memory requirements.

I want to be careful, here, not to lay blame for this at Apple’s feet. Doing true virtual memory requires hardware support. Microprocessors in 1984 didn’t have the full functionality required to support full demand paged virtual memory, so designing it into the original Mac OS would have been a waste of time. We often make design decisions that make perfect sense in light of current system limitations, only to have those design decisions come back to haunt us when Moore’s Law makes those systems orders of magnitude more powerful. There’s a reason Apple scrapped the Motorola 68K line of processors in favor of the PowerPC, not the least of which is the fact that it afforded them an opportunity to revisit some of those early design decisions.

Technical Achievement

Having reaped the benefits of a decade’s worth of Moore’s Law, we who now think very little of putting 128 MB or even a half a GB of memory into a laptop computer might find it difficult to grasp just how much of a problem the 68K memory wall presented for Mac Word 6.0. But we were trying to get the whole thing to run in 4 MB of memory—that’s total system memory, not just the application partition.

This was no small matter. Word 6 was getting a bevy of new features over and above Win Word 2.0. Relative to Mac Word 5.0, this was two major releases worth of feature changes. OLE, the built-in lexical analyzer and rule-based inference engine required for AutoCorrect/AutoFormat and a grammar checker that included state-of-the-art natural language processing technology (which made the grammar checker even more of a memory pig) combined with things like a full-blown macro language (WordBasic) to make Mac Word 6.0 huge relative to common Mac systems of that time.

Please note the “relative” qualifier to the word “huge” back there. To see this in perspective, fire up BBEdit on your Mac OS X machine, open the Terminal window, and type “top” at the command line. Now read the values in the RSIZE and VSIZE columns. When I open my .tcshrc file in BBEdit, those values are 12.1 MB and 164 MB respectively. As I type this document into my most recent build of Word 2004, those values are 36.6 MB and 222 MB respectively—and Word’s a full-blown word processor.

The amazing thing is that we actually managed to get Word 6.0 to run on systems that had only 4 MB of memory (well, “walk” might be a better word than “run,” but you get the point). To fully grasp the extent of this achievement, we need to understand a little bit about how programs are written and how they execute. What follows is my attempt to explain a fairly technical issue in lay terms. If your eyes start rolling into the back of your head, feel free to jump ahead to the next section.

Programs are written in relatively small chunks of code called “functions.” Each function represents a single, functional aspect of the program. Functions can represent high-level concepts (e.g. layout a page of text) or low-level concepts (format a single line of text within a page). Higher-level functions perform their work by calling lower-level functions, and there’s a protocol that helps the computer to know how to return from a low-level function back to the high-level function that called it. This protocol is known as “procedure prologue and epilogue” and it involves something known as a “call stack.” While the lower-level code is running, the higher-level code that called it is said to be “on the call stack.”

Trying to get a body of code to run in a memory space that’s smaller than the code itself involves something called code swapping. This is generally very easy to do if the code that you’re swapping out doesn’t cross these high-level to low-level boundaries. Our grammar checker is a good example. It represents a distinct functional unit, so we can swap the grammar checker’s code out of memory if we no longer need that code around without having to worry about swapping it back in when we’re done executing the current chunk of code.

But, we can group code at a level of granularity that crosses high-level to low-level functional boundaries. For example, the code that lays out a page of text can be in one module (or code segment), while the code that formats a single line of text can be in another module. When you’re laying out a single line of text, you really don’t need the code that lays out the whole page in memory. Conceptually, at least, you can swap out the page layout code while running the format line code.

There’s a problem with this idea: the page layout code calls the format line code, which means that the page layout code is still on the call stack. When the format line code is finished, the protocol that allows computer to know how to return execution back to the page layout code needs to know that the page layout code is no longer in memory. This is such a difficult problem that Apple’s documentation claimed that it was simply not possible to swap out code that was still on the call stack. Yet, this is exactly what we were able to do with Word 6.0

There is an unfortunate downside to being able to swap out code that’s on the call stack. It leads to something called thrashing. Consider our page layout/format line example. Page layout works by calling format line for each line of text on the page. Every time we cross the boundary between the page layout code and the format line code, we need to stop and load a chunk of code into memory, which will, in turn, require removing another chunk of code from memory.

Now, I’ve grossly oversimplified the whole process in order to explain what was going in. The swapping algorithm is a bit smarter about deciding what parts of the program to swap out of memory in order to be able to swap in a piece of code that’s needed immediately. In practice, then, it’s highly unlikely that page layout and format line would ever thrash by themselves. Nonetheless, thrashing does occur when the available memory is small enough. When the system thrashes like this, performance goes down the toilet.

Learning the Meaning of “Mac-Like”

OK, so Mac Word 6.0 was big and slow relative to the memory that most computers had available at the time we shipped it, but that’s not the reason why Mac Word 6.0 was such a crappy product, or at least not directly. Not long after Word 6.0 shipped, people could afford to add more memory to accommodate the added features of Mac Word 6.0. Those people who found those features to be very useful, and you’ll run into a few of them even today, felt that the cost of the added memory was worth the work-savings that those features afforded them. Moore’s Law, and the PowerPC, would have solved the memory problem in due time.

Moreover, while people complained about the performance, the biggest complaint we kept hearing about Mac Word 6.0 was that it wasn’t “Mac-like.” So, we spent a lot of time drilling down into what people meant when they said it wasn’t “Mac-like.” We did focus groups. Some of us hung out in various Usenet newsgroups. We talked to product reviewers. We talked to friends who used the product. It turns out that “Mac-like” meant Mac Word 5.0.

We spent so much time, and put so much effort into, solving all the technical problems of Mac Word 6.0 that we failed to make the UI of Mac Word 6.0 behave like Mac Word 5.0. As a result there were many differences, some little, some huge and even some that were simply gratuitous, between the way Mac Word 6.0 did things and the way Mac Word 5.0 did things. The end result was a UI that could only be described as clunky relative to Mac Word 5.0’s elegance. More importantly, Mac Word users had to unlearn all the ways they had come to do certain things, and relearn the Word 6.0 way of doing them.

My favorite example of this is they way you defined styles. In Mac Word 5.0, style definition was a semi-modal task. You defined or modified a style the same way you changed the font or paragraph properties in the document itself. In Mac Word 6.0, the task was completely modal. The entire array of menus and toolbar buttons that you could use in Mac Word 5.0 (and with which you were quite familiar as a user) was replaced by a single drop-down menu in the New/Modify style dialog box. Even today, you can’t use the Formatting Palette to change the font or paragraph information in a style in Word 2001 or Word X, and this remains one of the things I want to fix in Word before I leave MacBU.

The other thing we figured out as a result of coming to understand what “Mac-like” meant was that we weren’t going to be able to deliver “Mac-like” products if Office remained a singular product from which both the Win and Mac versions were built. The mere fact that “Mac-like” was an issue at all meant that there were some fundamental differences between the Win Word market and the Mac Word market. If we were to understand both those markets, then our Mac products and Win products needed separate marketing and PGM organizations. The lessons we learned from Mac Word 6.0 are some of the reasons that Mac BU exists today.

We still bang our heads against the wall from time to time. Understanding users isn’t an exact science. But, we do it far less often than we used to. And it really does feel much better. In a future post, I’ll describe in more detail how we go about trying to understand both our current users and potential new users.

As for my own role in Mac Word 6.0, I was responsible for the PowerPC port. But that’s also a story for another post.

Lastly, as for how I felt about the demise of Pyramid, all during the Word 97 project, I kept six empty Mac Word 6.0 boxes stacked in a vertical triangle next to my desk. I called it a slice out of a pyramid. A few people got the point. And the boxes when into recycling as soon as Mac BU was formed and we started work on Word 98.

Rick

Hungarian Notation

Rick Schaut — Sat, 14 Feb 2004 22:51:00 GMT

My dad always used to say, “Call me anything you want. Just don’t call me late for dinner.” If variables and functions in computer programs were sentient beings, I wonder if they’d say the same thing. As religious wars go among computer programmers, wars over naming conventions rank right up there with wars over favorite programming languages and favorite text editors.

Over the years, I’ve grown accustomed to various flame wars, and generally steer clear. But when I see flame wars over Hungarian Notation, I want to scream. You see, there are really two distinct naming conventions known as Hungarian Notation. I like to refer to them as Hungarian Notation and Anti-Hungarian Notation. The participants in these flame wars think they’re talking about Hungarian Notation, but they’re really flaming over Anti-Hungarian Notation. And that’s why I want to scream.

The difference between HN and A-HN is the story of two Charles’—Charles Simonyi and Charles Petzold. While I’m fairly versed in the Charles Simonyi story, I really have little knowledge of how the naming convention that Charles Simonyi outlined in his original paper metamorphosed into the naming convention that Charles Petzold described in Programming Windows. Perhaps Raymond Chen can fill us in on the details. They are, no doubt, sordid.

Whatever the details, they quite likely involve a misunderstanding fostered by an unfortunate choice of wording in Simonyi’s original paper. Not really Simony’s fault. After all, his native language is Hungarian, not English. While Simony talked about the “type” of a variable, he clearly wasn’t talking about the underlying language type of a given variable. He was talking about “type” in the sense of the operations that can be performed on a given variable. I like to refer to Simonyi’s concept of “type” as the “functional type” of a variable, as opposed to the variable’s “language type” as is often meant when people use the word “type” in conjunction with Hungarian Notation.

As Simonyi points out, a program written in C might have a function called, Position(int x, int y). The variables x and y, which both have an underlying language type of int, may well have distinctly different functional types. If x and y represent different dimensions in coordinate space, for example, it wouldn’t make sense to write, Position(y, x). The point, however, is that the variable names, x and y, conform to Simonyi-style Hungarian Notation.

Now, one way to metamorphose from Simonyi-style HN to Petzold-style HN, is to create unique user-defined language types for distinct functional types. In the above example, one might well say:

typedef int X;

typedef int Y;

void Position(X x, Y y);

Using this technique, we can maintain a one-to-one correspondence between the language type and the functional type, with the added benefit of being able to change the actual underlying language type of variables with functional types X and Y without having to perform a massive search/replace through all the code.

The one thing that Simonyi-style HN would not countenance would be the A-HN form of Position(int iX, int iY), because the “i” prefix doesn’t tell us anything about how iX and iY are used. It’s pointless information—decorations added by someone who really doesn’t understand the point of Simonyi-style HN.

For those of us who have grown accustomed to Simonyi-style HN, the whole “i” means “int” prefix thing is doubly horrible, because the “i” prefix in Simonyi-style HN is commonly understood to be an index into an array. If I put a character into an int, I’m not going to call it iCh. I’m just going to call it ch. To me, an ich is an index to a character in an array of characters.

Now, I’m a Word guy. Word has device-independent layout, where text layout is done in a coordinate space based on the design units of the font, which means that we often have to convert between layout coordinates and window coordinates. Being able to type XwFromXl sure beats the pants off having to type something like HorizLayCoordToHorizWinCoord.

But, I’m not here to add to the flame war. I’m just trying to correct a wrong that’s been allowed to fester for far too long. And, no, don’t get me started on MFC. Let’s just say that a good number of people who managed to not understand Charles Simonyi’s original paper happened to work at Microsoft.

So, now the .NET guidelines are steering people away from using Hungarian Notation, or more accurately Anti-Hungarian Notation, quite likely because it’s too late to correct years of bad practices. Better to just cut one’s losses. That’s something of a shame, because Simonyi-style Hungarian Notation was the basis leading a certain Excel developer to devise such function names as CleanPots and FreePot (a POT being a “piece ‘o’ text”).

Rick

Ed Fries

Rick Schaut — Fri, 13 Feb 2004 05:22:00 GMT

Ed Fries is retiring from Microsoft, and last night we held a little shindig to send him off

Ed Fries is retiring from Microsoft, and last night we held a little shindig to send him off. Ed’s accomplishments as head of the games business unit are pretty well known and impressive. During his eight-year tenure, he shipped over 120 new titles, of which 18 sold over a million copies. Were he in the music business, that would be 18 platinum CDs in 8 years. Ed was also responsible for anything having to do with Xbox.

Before Ed went off to run the games group, he was development manager for Word. Before that, he was a lead developer in Excel. He was also one of the co-founders of Tom and Ed’s Bogus Software, purveyors of the original “Fish!” screen saver. Their motto was, “A fish on every race track.”

Ed invented DDE, and, as proof of concept, he wrote a program that would send DDE messages to the Fish screen saver. The DDE messages would tell the screen saver to create submarines that would shoot down the fish.

Over all these things, however, I’ll always associate Ed with the infamous “Swing around the Wing.” Ed used to set up miniature golf courses once a week in building 8. To play, you’d send Ed an e-mail, and he’d group you up and send you a tee time.

Of all the various stories I could tell about this, none is quite capable of conveying the relaxed nature of the “Swing around the Wing” than the story of the invention of the Osgood scale for measuring the depth of a ding in the wall.

Osgood was an intern, who stepped up for his first shot, took a fairly hefty back swing, and toed the ball severely, at which point the ball rocketed off at a 90 degree angle from the direction of his swing and embedded itself in the wall next to him to a depth of nearly half the golf ball. As Ed says, it was almost perfection.

Since then, a ding in the wall was always measured in Osgoods, well, fractions of an Osgood. I don’t know of anyone who left a ding deeper than even half an Osgood. The most common depth was 1/8^th of an Osgood.

Yeah, I’ll miss you, Ed. And don’t forget the fish.