Mac Mojo : Development

Update on Office 2008 progress!

Geoff Price — Tue, 09 Oct 2007 02:08:00 GMT

Last Wednesday was our general “ZBB” target for the Office 2008 project, a major milestone on the road to release and something we’ve been pushing hard for over the past several months. We saw some fantastic surging by the development team to clear out backlogged product issues late into Wednesday night. Thursday, we sat down to review project status and metrics in depth, and to hear all teams report out on their plans and readiness to lock down for release. The bottom line takeaway that you may be most interested in: all teams have confirmed readiness to ship on the current schedule. As previously announced, this means RTM in December and general availability starting on January 15th (by region).

ZBB what? This stands for Zero Bug Bounce, or as alternately phrased, Zero Bug Backlog. At this point we have been logging, tracking and verifying all changes to the product in great detail for some time – collectively we refer to this list of logged issues as the “bug list”, though in reality it includes a variety of issues including bug/defect reports, tracking records for artwork or content, usability improvements, numerous suggestions (submitted by team members or beta testers, or representing customer or partner requests), and various other categories of issues. The ZBB milestone is defined as the date across which we will no longer carry logged product issues that are more than one week old. Hence, the “backlog” of issues has been cleared out, and all older pending decisions on what we are or are not going to change before we ship have been made. It also means that the developers have “caught up” or “outpaced” the incoming find rate of our test efforts.

From here on out, the focus is increasingly on lock down. We are testing for defects literally around the clock, using a variety of methods ranging from automated efforts such as “massive file testing” to targeted manual test passes and “ad hoc” free testing. We also continue to log and fix significant issues found in private beta testing. At this point we remain busy fixing bugs as they are found (any one of our millions of lines of code could be the source of a defect,) but there will now be increased scrutiny on making changes – i.e., is there sufficient customer need to justify a proposed code change given the associated risk of regression (the outside chance that making a given change causes a new and potentially more severe problem.) Stabilizing the product for release requires conscious commitment to slowing and eventually stopping the changes we’re making to it, and as a result we now start reviewing all proposed changes daily in “triage”.

I’d like to congratulate and thank everyone on the product team for an amazing effort on ZBB – our developers, testers and program managers have all done incredible work. I’d also like to thank every other person who has helped in the creation of Office 2008: planners, researchers, designers, artists, writers, editors, builders, product managers, recruiters, HR generalists, international teams, MVPs, beta testers, partners, vendors, contractors, our many friends at Apple, partners across Microsoft, and many others have all helped us get this far. Finally, a very sincere special thanks to all of our families and friends for their support during this push.

Collectively we’ve made nearly four years worth of heavy investment in modernization, improvements and features in Mac Office. We look forward to continuing to share more about the new suite here on the blog and out in the world straight through RTM and into next year. But, as folks like to say, at this stage of the project the most important feature is shipping.

We’re getting more excited about that feature every day.

Silverlight Excitement

blairn — Wed, 23 May 2007 21:24:00 GMT

Are you excited about Silverlight? I am.

In case you haven't heard, Silverlight is a new rich internet application platform developed by Microsoft with support on both Windows and Macintosh systems. It's pretty awesome. The Silverlight demos presented during the Mix 07 conference keynote were simply out of this world, and featured the cross-platform experience on Mac (or, depending on your perspective, the cross-platform experience on Windows) quite nicely.

Developers can use XAML (a subset of the same XML-based vector graphics markup language used by Windows Presentation Foundation on Vista) in the browser, and bind XAML controls through the DOM to either Javascript or the .NET Framework core components, the latter of which was brought over to Mac to help make this happen. That's right; I said .NET Framework on Mac. Exciting? You bet.

Too geeky for you? How about streaming Windows Media audio and video, supported by Microsoft, including HD video up to 720p? Check out these Mix 07 demos (I'm personally a major fan of Top Banana) and you'll start to get a sense of the kinds of exciting rich internet experiences that Silverlight helps to enable.

Not geeky enough for you? What about dynamic language runtime support, so you can implement all your code-behind in Ruby, Python, or what have you? How about designers and developers working together with seamlessly integrated workflows to help each other deliver striking content with killer functionality behind the scenes? How about the power and performance of the core .NET Framework helping to deliver both software and services online, across browsers, and across platforms? How about 4 GB of free Microsoft-hosted video? How cool is that?

Did I mention that Silverlight is supported in both Safari and Firefox browsers on the Mac? Okay, I admit, I'm feeling just a little bit of Silverlight excitement.

So, that's all good and exciting, but what does Silverlight have to do with MacBU anyways? Well, in a sense, not much: Silverlight is not a MacBU effort. Why not, you ask? Well, as you probably know, MacBU has never been the only source within Microsoft for software on the Mac. We certainly are the center of gravity for exciting Mac-specific experiences at Microsoft, but Silverlight (which is really about exciting cross-platform experiences) doesn't really fit that "Mac-specific" charter. But we're certainly excited to have Silverlight join us here on this exciting platform that we call the Mac.

Check out Silverlight for yourself.

Get converted

Geoff Price — Tue, 15 May 2007 20:56:00 GMT

As previously announced, and as indicated by the lower frequency in blogcasting here, everyone at MacBU is exceedingly busy completing Office 2008, which is on track for release later this year. This work is our primary focus, and is the largest undertaking in our ten year history as an independent team. We are developing the product for versions beyond 2008 with broad investment in native Mac OS X architecture as well as adoption of a new, next generation document format: Office Open XML.

We've made great progress, and as previously promised we're releasing some of this new functionality in a form that you can start using right away: Beta release #1 of the Microsoft Office Open XML File Format Converter for Mac is now available for download.

This is a stand-alone Macintosh application that converts .docx documents - that is, documents saved by Word 2007 for Windows in the Office Open XML file format - into rich text format (RTF) documents so that they can be automatically opened in either Word 2004 or Word v.X for Mac OS X.

With this free converter we passionately want to get you up and reading the new documents you are receiving. We do not, however, want to see you inadvertently mess up any critical documents you are working with. For that reason, only one-way (read only) conversion is supported in this beta. When sending documents back to colleagues and contacts, we recommend saving to the default .doc format from Mac Word (listed as "Word document" in the save dialog). Similarly, we continue to recommend that you advise friends and colleagues who use Office 2007 and collaborate regularly with Mac users to save their documents as a "Word/Excel/PowerPoint 97-2003 Document" (.doc, .xls, .ppt) to ensure that the files can be robustly shared across platforms while waiting for final availability of Office 2008 for Mac.

Personally, I'm getting plenty of opportunities to use this new converter right out of the gate. Office 2007 for Windows is pretty widely deployed across the numerous PC desktops we have here internally at Microsoft. So far the converter is doing great, but then in most cases I tend to only need to read and review the documents rather than co-edit them (product plans, service agreements, draft letters, staff communications and the like). For active collaboration on critical or complex documents - e.g. where getting the formatting, tables etc. correct is essential - you'll want to continue to work with the older (.doc) format from the start.

Why a stand-alone converter application? We chose this route because it supports both Office 2004 and Office v.X for Mac users, while providing some bonus functionality such as batch conversion, and also keeping the team focused on Office 2008 by avoiding major test efforts around invasive changes to our shipping Office 2004 product. The user interface for the converter actually began life as an internal test tool (where you can imagine the batch processing comes in handy) - it was so slick it quickly became obvious that it was a great solution for the converter. You can drag and drop files onto the converter icon or application of course, but can also simply double-click .docx files to invoke it. The first thing I did is turn on this preference to immediately open a newly converted document in Word, so that I can skip the step of finding the new RTF document and dragging it to Word.

Why Rich Text Format? RTF (once called the "interchange format") is simply a highly convenient intermediate format for the beta converter to use; I'll let one of our Word experts (like Rick) expand on the technical reasons why this is the case if there's interest.

We plan to release a final integrated converter for Office 2004, which will appear as an update that allows you to simply open and save the new file formats as if they'd always been there (though, some of the newer functionality expressed in the formats will naturally only be available in Office 2008). We are on track to deliver this final integrated converter for Office 2004 six to eight weeks after Office 2008 for Mac is available. We also will release a final version of the stand-alone converter soon after.

Why wait until after we ship to release the final converters? The converters include and rely upon the same new code that lives in Office 2008 (so you're getting early access to some of the new code today - running natively as a Universal Binary, of course). As the code improves in the applications, so will the converters improve, and as a result the converters will not be final until Office 2008 is also fully complete and fully tested.

Incremental updates to the beta converter will be coming this summer, including those to roll in PowerPoint and Excel document support. If you have installed the new beta converter, these updates will be pushed out automatically through the Microsoft AutoUpdate service.

Congratulations to the product team here in MacBU on the great progress made as we approach the finish line on Microsoft's best ever release of Office for the Macintosh - Microsoft Office 2008 for Mac - and a big thanks for your hard work in building the converter.

P.S. If you are interested in learning more about the Office Open XML formats and the ways third parties are adopting and plugging into them, Brian Jones' blog is a great place to start.

Converters Coming! Free and (Fairly) Fast.

sherjo — Tue, 05 Dec 2006 22:11:00 GMT

There's been some pretty alarmist news stories this morning (apcstart.com; CrunchGear) about file format converters and Office for Mac. Because some of these stories seem designed more to inflame than inform, I want to reiterate our previously reported intention to provide free converters, clarify our timing for their release, and also give some direction on how best to avoid incompatibility until they are available.

Be it resolved: The Mac BU WILL issue free, downloadable file format converters that allow users to read the new Microsoft Office Open XML Format. We announced that publicly at WWDC, and nothing has changed.

Timing: We’re working on our file format converters as I write. We had to wait until Office 2007 bits and the new file format itself were locked down. During this time, we spent the last year and a half prepping and planning for our own development of file format converters for Office for Mac. This included the basic supporting work of a rich and compatible XML parser, code to understand the new package structure, and beginning work on reading and writing early development versions of the file format. So now that Office for Windows has been released, we are working on completing compatibility with the released formats, while also completing other major work such as moving our codebase to the Intel platform, which we have discussed at some length on this very blog and elsewhere. We are running on target and expect to release a free public beta version of the file format converters in Spring 2007, with final converters available six to eight weeks after we launch our next version of Office for Mac (which, as previously reported, will be available 6-8 months after general availability of Win Office.) The next version of Office for Mac will natively read the Open XML Format; users of the current version of Office will have converters in order to maintain compatibility with the new Office for Windows.

There will be a delta between general availability of Win Office (January) and converters from MacBU (expected late March/early April.) We realize this will be an inconvenience for some of you (trust me, we know - 90% of Microsoft has been dogfooding Office 2007 for many months, and we in the MacBU are well used to asking for down-reved versions ourselves). For now, we recommend that Mac users advise their friends and colleagues using Office 2007 to save their documents as a “Word/Excel/PowerPoint 97-2003 Document” (.doc, .xls, .ppt) to ensure the documents can be shared across platforms.

Living complexity

Geoff Price — Fri, 10 Nov 2006 23:16:00 GMT

Brad's Carl Sagan-style post on code size stirs up some familiar questions about complexity management here at MacBU. With every release of Office, we remove, rewrite and add new code. Often we can remove significant amounts of code without losing functionality – say, remove use of a custom HTML renderer when the OS offers a built-in one instead. I recall how many of us took it as a badge of pride during the Carbon transition of Office v.X that after a year of intensive coding our net contribution in terms of new lines of code was significantly negative. Those were the days, hacking and rewriting. But of course, as the total functionality of the suite increases over time, the total volume of code used to build it tends to inevitably follow. There has been a lot of functionality added in twenty years.

Is it necessary? It's pretty easy to find people who agree that Office has more functionality than they personally need. Where the agreement tends to fall apart is when you ask what the first feature to be cut should be, at which point one person's "bloat" is another's "only feature I use every day". It's also nice to know the functionality and compatibility is there, as you find the need or the advantages of it. I use all of our apps every day, not to test but just to get work done – Exchange-based mail and calendaring in Entourage, analyzing data and lists in Excel, IMing over corp chat – it's an interesting job being able to use the software every day to talk about the job of making the software. My Office skills were pretty minimal when I got here, but (and some coworkers might smirk at this) they've come a long way. You may not have heard of (say) pivot tables, but once you start using them for certain kinds of analysis you can't imagine how you ever lived without them.

Of course, our goal isn't to just pile features on top of features, but to harness the increasing power in the suite with an effective and focused user experience that puts functionality in reach and in context. In some cases that can be pretty easy, but generally it's actually pretty hard, and something we have to think about with everything we do.

Is it slow? Generally, there's no linear relationship, where adding 10% code means everything is 10% slower. There's quite a lot of engineering happening down at the metal to swap in the currently executing code as the application runs. The problem is in how much work the application is trying to do – if it's trying to do a lot, it's necessarily going to take longer to do. When we work on performance bottleneck areas (such as boot speed) that is generally what we are looking at, i.e. do we really need to be doing all this now? For this release, running native on Intel is orders of magnitude more important for speed than trying to trim raw code size, for example.

Is it sustainable? It's important to understand that while the code has been written and expanded over the past twenty plus years, it has also been tested and validated for twenty plus years. Insane amounts of testing. Stabilized or "baked" code provides a baseline for correctness and quality – as new changes introduce unintended problems (regressions), we know we just need to hunt down which new change is the culprit and understand how it needs to be redone. The real impact is on productivity: while correct, that baked code can vary wildly in how well factored it is (well factored being the opposite of "spaghetti"), and therefore how sensitive to regression. We definitely have some pasta mixed in as part of our overall diet, and we have to develop strategies for dealing with that. Sometimes it's best to simply say "do not disturb the water", but that's not always an option (say, for example, you decide to blow out the Excel cell table.)

Shouldn't we just rewrite it all from scratch, and "do it right" so that *all* of the code is perfectly well factored according to modern sensibility? JoelOnSoftware made some great observations on that topic in this article: Things You Should Never Do, Part I. (I believe link-master Brian sent that to me originally. Pyramid was before my time here so I'll leave it to others to comment on whether Joel's observations on that project are fair and accurate or not.)

Some people keep souvenir copies of tech editorials from ten years ago predicting that Microsoft will be unable to release any further versions of Excel, because the code base has simply gotten too big and Microsoft will have to rewrite it from scratch. That was quite a few releases of Excel ago.

Joel points out that code doesn't rust, but the complexity problem is real because massive change is inevitable if you are trying to keep your software on the leading edge – especially on the Mac. We plan to continue shipping Mac software into the foreseeable future, so it is easy for us to make major architectural decisions with the very long term in mind. Where it gets hard is balancing this against short term customer need – what resources can we apply to deep rearchitecture or local refactoring, given that these resources will not be available to solve the most pressing problems that are actually visible to customers? While we made significant investments in future capability earlier in the current product cycle, right now that customer need is pretty extreme and we're 100% focused on getting this release out into the market.

Finding this sort of balance overall, and dealing with this level of complexity in general, is clearly our defining challenge with Office. It takes some time to get your head wrapped around it all. We might go days without seeing a particular new developer, and we'll have to send a veteran search and rescue squad into their area of the code to pull them out. Of course, it doesn't just impact developers, it's in the density of the test cases, the maintenance of the automation, the range of functionality and interactions our designers have to understand, the scope of documentation, the pace of change. Managing complexity is where we live. It's not for everyone, but fortunately I find it all pretty fun at the end of the day... gives my head something to do.

It's all in the numbers ...

ExCntx — Fri, 03 Nov 2006 02:09:00 GMT

A friend of mine and I were shooting the ... breeze (freaking ‘G' ratings for blogs) and he was asking me when the next version of Office was coming out. Now being the wise Microsoft spin doctor that I am, and having spent numerous hours being brainwashed by marketing, p.r. and some guys in black who look like Jessie Ventura and Alex Trebek (anyone get that reference?) I looked him square in the eye and asked, "How many lines of code are there in Mac Office?"

Not to be caught off guard, he stared right back and asked, "You're going to kill me after you tell me, right?" I smiled that Cheshire cat smile I have and said, slowly, "Maybe ...." I let the pause go on for a bit longer than normal and then laughed.

So I changed the subject to movies and after a bit he asked me, "How many lines of code are there in Mac Office?" I looked him and asked, "Do you really want to go there? " He nodded as he moved his lawn chair a little out of arm's reach.

About 30,000,000 lines of code make up the current version of Office that we are developing. That's no typo; in fact, I had to figure that out because of a research project I was working on ... if I told you what it was, I would definitely have to ... oh, yeah, ‘G' ratings ... anyway back to the train of thought.

It's really mind blowing when you think about it. Each developer is responsible for, on average, about 428,000 lines of code. Some people have more areas, some have less, but if you just think of that number, that's a tremendous amount of responsibility per developer. Consider for a moment also that there are probably 10x (yes ten times) or more developers on Windows Office, although I don't know the exact count.

Just to throw another mind-numbing number out at you, there are about 40 lines of text on the page of an average paperback book. That means one developer is responsible for about a 10,700-page book. Or if you break it down smaller, if the average paperback is 300 pages, that means each developer is responsible for about 35 or more paperbacks on his desk. Imagine trying to find a single typo in all those books -- that's what most bugs are, don't ya know!

Rick might have a better guess, but from what I have seen the average life of the code (before this version), was about 12 years, meaning most of the code was written back when 68k and 386 processors ruled the world. A5 bring back any memories for people? What about segmented memory? Why is this important? Read on.

The final killer number that I think is important in answering my question is 50%. What does that number mean? It represents the amount of turnover we have had in our developer organization in the past two years. Let that sink in for a bit: during these past two years we started the move to XCode, then started the move to Intel, brought on about a bunch of new people and asked each of them to learn somewhere on the order of 400,000 lines of code. Most of which was written before some of these new hires started high school! Either that, or they started work on a brand new feature and we pushed all the responsibilities of that code onto people who never saw it before or were already working on other things.

I guess the short answer to my friend's question is that the next version is coming out as fast as we can develop it. Everyone in the MacBU, developers, testers, UA, PM, Loc, and managers are trying to make this the best version of office for both PowerPC and Intel based Macs. But as we have said before on this blog and will continue to say, we're making progress, staying on track and hoping to get this puppy wrapped up and shipped off to you as soon as possible. But for now we're all trying to get through our 35 books trying to find those blasted typos.

Oh, and for the record, I think I have about 50 or so books on my desk ... some are written in OLE ...

Closet space, the final frontier

Office for Mac — Tue, 24 Oct 2006 01:27:00 GMT

The last event of C4 was a visit to Chicago's Adler Planetarium, with a special behind the scenes (and beneath the stairs) tour of the computer room that runs the video conferencing room and the CyberSpace classroom. Well, "room" is charitable. "Closet" would be better. "Crawlspace", "pit" or "graduate student living arrangement" would also work.

Contrary to the skeptical whisperings of some of the Planetarium staff ("Why are you showing them closets?"), the highlight was easily the rack of more than two dozen G4 cubes wired up as the brains of the classroom workstations.

I find great irony in the fact that a machine which was originally designed more as an objet d'art than a computer turns out to be a very useful computer for a museum, and a group of programmers come to appreciate the techno-beauty of the installation, even if it is in a closet.

I'll leave you with a few classic quotes from C4, as best I can remember them. It's been great being your weekend guest-blogger. Thanks for reading.

- Olof

"It was like an unwritten rule, except that it was written."

- John Gruber on the Human Interface Guidelines

"Who'd have thought I'd be up here saying how wonderful NavServices is?"

- Brent Simmons

"It was like making apples from applesauce."

- Brian Fitzpatrick on writing cvs2svn

"Before MySpace, we had friendships.

Before NBA Live, we had basketball.

Before Second Life, we had life.

Before Delicious Library, we had shelves."

- Aaron Hillegass on the importance of solving real problems

"Subversion is not a big truck."

- Brian Fitzpatrick

Checking out and checking in

Office for Mac — Tue, 24 Oct 2006 00:56:00 GMT

C4 is over and I'm back in Seattle, which means these last two posts will be Almost Live-blogging C4.

I get about 60 emails every day from a robot. Well, not a robot that moves around, but our version control system -- the server that manages our project's source code. Every time someone checks in a change to the source code, the system sends out email to everyone that includes information about what got changed, who changed it, and their comments about that change.

MacBU is somewhat schitzophrenic in that roughly half of our development team is in Redmond, and half in Silicon Valley. I'm in Redmond, so when I started at MacBU, I got to know the SIlicon Valley part of the team primarily via email, a good deal of which was from the source control system. And there are also smaller parts of our team in Ireland and Japan and China where the email communication is even more important.

Our system is set up this way so as to encourage good collaboration. For example, knowing that everyone will get email that contains the checkin comments, most people try to write decent checkin comments. This good for team communication in the present, but it is also crucial for helping our future selves, when we'll look back on our checkin comments from today and try to answer the crucial question "What were we thinking?".

But 60 emails a day is overload. I usually scan them but often don't have the time to follow closely what people have been working on. They are still helpful, of course, but the lessons are clear -- our software systems help us communicate within the team, and thereby build team cohesiveness and awareness, but there are obvious limitations.

So what got me thinking about this? Brian Fitzpatrick, one of the people behind Subversion, the latest and greatest open-source source control project, spoke at C4 on Saturday. Brian had a number of examples of how the features available in a version control system affect the community that uses it, and he talked about trying to make decisions about the design of Subversion with the community dynamics in mind.

Interestingly, its not always obvious what the 'right' choice is, and indeed, there are cases where one project might want a certain feature done one way, and another community might want a completely different implementation. For example, the very meaning of 'checking in' might be very different. At MacBU, checking in means taking responsibility that the code works and doesn't break the existing build. But in another setting (think Wikipedia), checking in could mean "here's my ideas for what this file should be -- what do the rest of you think?".

The trend of thinking about software not just as a set of features, but instead as a foundation for community building isn't a new one, but it still surprising how far reaching and subtle the influences can be. Working on the Mac Office products, we use the phrase 'personal productivity' quite a bit, and most people know really well what makes themselves productive.

We also care about 'team productivity' and 'collaboration', but those things can mean different things to different people, and they are much harder features to deliver, especially when 'personal productivity' can be at odds with 'team productivity'. It's more evidence that there's plenty of innovation still needed in our market segment, and why there's a future for Mac Office well beyond the horizon we can see today.

Film at Eleven

Office for Mac — Sat, 21 Oct 2006 20:14:00 GMT

It's Olof again, liveblogging from C4.

John Gruber took us on a tour of Apple User Interface last night, and talked about consistency and uniformity. But he really got the thought provokage started with the observation

"Designing an app is like directing a movie."

There is a building wave of people thinking of software in cinematic terms. Do you remember seeing the genie effect for the first time? Functionally, a window gets collapsed into the dock. But if you were like me, you played with the genie effect a few times just to see it.

Whether or not the genie's swoosh motion into the dock really does help the user by, it was a defining part of the user experience. For Mac developers in 2006, the idea of 'software as movie' is taking hold. If you haven't heard the phrase 'cinematic experience' enough this year, you haven't been drinking enough Kool-Aid.

And increasingly, application developers have more and more tools available to make animation and high quality sound a part of applications. We've got compositing windows and serious CoreImage graphics processing, and computing power to spare to waste on pixel candy. There are more subtle hints, too. 'Spotlight', 'Sherlock' or 'Time Machine' as names for technologies feel very Hollywood. But at what point does building an application turn into directing a movie? And is that a bad thing?

Gruber offers Delicious Library as an example of an application where a big part of the Wow factor was an only marginally useful but stunningly gorgeous library display shelf:

"If I was a competitor to Delicious Library, I'd be demoralized looking at it. Here I thought I was writing software and I got beat by a movie."

Continuing in that direction, its easy (but perhaps scary) to imagine a world where applications are differentiated like movies are. Easy because Apple ships examples like GarageBand and iTunes that are moving in that direction. Scary because there are examples like Kai's Power Tools.

Scary, too, because it means software developers need to think more and more like filmmakers. Maybe MacBU needs to hire some people with movie industry experience. What's that? We already do? Cool!

C4 is Teh Bomb

Office for Mac — Sat, 21 Oct 2006 05:04:00 GMT

Hi - I'm Olof. I guess I'm the first "guest blogger" on MacMojo. I'm a developer on the Mac Office team, and for the weekend, I'm in Chicago at C4.

I have a really strong sense of what C4 is going to be like, even though I've never been to one before. In fact this is the first one. There is no precedent for who will come, what the atmosphere will be like. There's no commercial entity driving it forward like Apple at WWDC, no hip branding to overhype the experience (sorry, Wolf's tie doesn't qualify).

Why do I have such confidence about the weekend experience ahead? Trust. Relationships. Peer-to-peer. That is, I trust Jon to put on a good show, I value the people I meet at an event like this, and information from peers about technology is completely different from information from technology promoters.

I met Jon "Wolf" Rentzsch when he had just started up the NorthWest of Us Programmer's SIG. He was a lot less Wolf-y then. He cared about people and technology. We talked shop. Extolled the virtures of scripting, of WebObjects, of source control. He knew cool things about threading. But more than anything, the Programmer's SIG was a time to revel in cool stuff. He made walking to get oily disgusting pizza an exhilarating event.

When MacHack ceaased to be, a lot of us felt a void -- and I'm hoping -- expecting -- this to be something to fill that space. Apparently, a number of other people feel the same way. I hear a lot of people disappointed that they couldn't make it. Did they have the same preconceptions about C4 as I did?

Talking with folk here, it seems like they most value the community that they expect will be there. Who is that? People who trust Jon to collect an interesting group of people and organize a fun time.

So far, so good. At dinner, some people are having problems with NetNewsWire, and Brent Simmons is providing tableside tech support. Too cool.

And, Wolf doesn't disappoint with his keynote. Lots of Web 2.0 mockery mixed with Apple developer geekery. And just like I expected, Wolf delivers on all my preconceptions. He wants C4 to be a sponsorship free event, high on community bonding and honest technology discussions. And he shares the secret to the Cheetah-era BombApp.app, bringing down the house with a demo of how to reimplement it in Cocoa with 0 lines of code. More soon.

And I do hope I can figure out how to do hyperlinks on these posts. C4 needs a link. So does Wolf's tie.

Office Delay? Ya Don't Say.

sherjo — Thu, 12 Oct 2006 03:27:00 GMT

No, seriously, you don't say (or shouldn't), because it isn't true. Over the last few days, some Mac sites have been reporting that the Universal Binary version of Office for Mac (officially unnamed, but currently code-named Office 12) has been delayed, but there is no delay or deviation from our development schedule. We're hitting our milestones, checking in our features, and making the move to Intel as planned. We've totally moved from Code Warrior to Xcode, so we've crested that hill. We usually ship 6 - 8 months after the availability of Office for Windows so we can do compatibility testing. This has been our shipping cycle for ages, and we're right on track. In fact, for Office 12, we've not even officially announced a launch date (but when we do, we should do it here first).

Incremental mojo

Geoff Price — Fri, 06 Oct 2006 18:52:00 GMT

Hello, another introduction for you – my name is Geoff Price, and I’m the Product Unit Manager for MacBU in Redmond. This is MS speak for saying I have responsibility for the core engineering teams here (development, testing, program management and user experience research.) I look forward to contributing to this eclectic RSS broadcast at irregularly scheduled times, and I’d like to echo those who have gone before in thanking you very much for tuning in.

I intend to cover more specific topics about our software, our teams, what we build and why, but first I thought I’d start out (perhaps uncreatively) by filling in some of the background picture on my particular history of development on the Mac and how I got to be here.

My programming experience with Apple computers began on the Apple IIe in 1983, and my development experience with the Mac itself began around the time of the introduction of the Mac II in 1987. (I realize it is possibly more fashionable these days to date your involvement with the Mac all the way back to at least 1982, or some other date even earlier than its actual introduction, but these happen to be the correct dates in my case.)

Back in the day, I worked at a Mac lab in U.C. Santa Barbara, and when student traffic was slow I could use the time to do coursework, experiment with Turbo Pascal, or write political columns for the campus paper (random aside: one of these concerned the case of an amateur talk radio host by the name of Sean Hannity, facing dismissal from the campus airwaves over inflammatory on-air remarks. A story, I suppose, for another blog.)

I soon joined a team operating out of the deeper recesses of the lab that worked with faculty to explore the practical teaching and learning opportunities emerging out of the new “multimedia” technology. In order to rapidly develop courseware applications for interested faculty, this entailed a deep immersion in the enchanting, “hyperlinked”, quasi-object oriented world of HyperCard, which would prove to be a consuming focus for the next several years.

As long-time Mac enthusiasts know well, HyperCard tended to stir fierce devotion in its supporters. At the time, it captured a sense of the creative possibilities of computing, as well as the spirit of the Mac in enabling the rapid development of graphic user interfaces with a more accessible and English-like interpreted language for programming. It also had a bit of the punk sensibility of DIY (if with a highly geek cast.)

This was at a time when Apple was pretty dominant in education, and there was terrific momentum behind HyperCard there. Now, HyperCard was relatively accessible, but it still took some learning and a lot of scripting, and by default had some limitations, especially relative to multimedia. It was however highly extensible, as developers could build compiled code resources (XCMDs) that tapped into any underlying system API, which could then be accessed as commands in the HyperTalk scripting language.

My work on courseware applications led to a partnership with a teacher named Mark Ferrer, who had ideas about trying to provide teachers and students a richer framework for developing their own interactive multimedia applications. The result of this was a HyperCard-based authoring product with a name rooted in academic workshops, HyperGASP, which we eventually released as a small-scale commercial product, touring around schools and colleges and working with interested educators.

One thing this initial foray into commercial software made me appreciate pretty clearly was the amount of work involved and the value of specialized expertise in all aspects of software development. I designed UI, wrote code, tested it, wrote documentation (in a stack, of course), spent time working with reviewers at tech publications, mocked up ads in Photoshop, managed beta lists, handled tech support, etc. You can still read about the product here, conveniently and cryonically preserved on the web. (Funny to go back and read some of that. I forgot about the great Word integration features we threw in – so Microsoft-friendly!)

Ultimately of course, HyperCard didn’t make it. Apple struggled with the business case, the rise of the PC in all segments as well as alternative authoring environments displaced a lot of HyperCard seats, and ultimately the cross-platform standard of the web provided a much more killer framework for hyperlinks. Many creative endeavors that had incubated in and around HyperCard lived on, such as wikis, boingboing, and Myst.

(Quick test for HyperCard gurus lurking in the audience: given that you could only paste cards after the current card, what’s a one line command you could type into the message box which would move a given card (say #n) to the first position in a stack, and why does it work?)

My personal life got busy with the arrival of my first two children. I transitioned to working for a publisher in Santa Barbara who had carried HyperGASP in their catalog, helping them build custom cross-platform multimedia tools. In addition to Mac programming this entailed ramping up on more serious Windows application development (Win32/MFC) as well – something that seemed like a reasonable hedge given the share Windows 95 was establishing.

By the time I dropped an updated résumé off with a recruiting agency at MacWorld Expo, I was resigned to the idea that my next job might end up being straight-up PC programming, or something completely different like Java. Surprisingly, the company that turned out to be most interested in the Mac portion of my résumé was, of all companies, Microsoft.

It was 1998, soon after the formation of MacBU as an independent business group, and the new team was staffing up on Mac-focused developers. The existing roster included devs with a wide range of interesting experiences in the industry (one senior developer had previously worked at Apple on HyperCard’s file format, among other technologies.) The opportunity to work on a well-supported project with a group of really smart folks gathered from all over the world – as well as Excel architect Rick Powell’s persistence – eventually got me on board.

At this point I’ve been with MacBU for over eight years, starting as a developer in Mac Excel, where our first challenge was to move our development environment and code from MSVC to CodeWarrior so that we could move forward on the platform. I was the Excel dev lead through the Carbonization effort of Office v.X, rewriting the older and thicker layer of Excel’s event handling nervous system to be based on the native OS X (and more preemptive multithreading-friendly) APIs of Carbon. Since 2002 I’ve been focused on managing first the development team and now the Redmond engineering teams as a whole, which among other things means doing what I can to maintain an environment where smart people will continue to want to come and work.

I’ll have to save some stories for later posts, as this “brief” introductory post has already gone long, but I’ll close up with a short state of the union.

One nice thing about MacBU is that things are certainly never dull, working as we do at the intersection where Office as a powerful and constantly evolving standard for “information work” meets the rich and rapidly evolving platform of the Macintosh. This combination entails ongoing engineering challenges, which have proven particularly intense for the release we’re building now. We have major work underway for Intel-based hardware, significant rewrites of user interface code to run in composited HIViews, and a cross-platform push to provide completely revamped and more open file formats, as a sampling of “under the hood” changes alone.

The good news relative to all this is that, while it’s taken time to grow and sign up the caliber of people we require, the group of developers we have on board now is the largest we’ve had in MacBU’s history, with a lot of fantastic people on board. Our test team has geared up with the most robust arsenal in terms of test plans and automation we’ve ever put together as well, and is at work validating deep quality as major functionality gets checked in. That said, we’re still looking to grow further, and I’d be remiss if I failed to mention that we’ve recently created new openings in both dev and test here in Redmond (and also have openings on our engineering teams in Mountain View, all of which you can review and/or respond to here.)

So, folks are downright busy and we still have a ways to go before we can share a whole lot more about new features in Office. In the meantime, we’ll try to keep the blog posts coming. Again, welcome and thanks for tuning in.

On the power of a small team -or- Why MacBU testing rocks

joeleblanc — Sat, 30 Sep 2006 03:14:00 GMT

Geez, now I have to follow up David's awesomely funny entry about the food alias with my dry post on testing.

One of the first classes I took after starting here was called “Testing at Microsoft for New SDETs.” This class was full of about 25 people who had either just started with the company, or just started a new job as an SDET. (SDET here means “Software Development Engineer in Test) A good portion of the class was very applicable to what I would end up doing, although a lot of it was focused on Windows technology. Some of the more applicable parts related to test planning, test cases, and more of the theory of testing that applies to any platform you’re working on. During lunch I was talking to the instructor about what I would be working on, the MacBU, and how we might do things differently than the rest of the company. When I told him how big the team was, he was shocked. “How can you test with such a small group and ensure a quality release?”I told him, honestly, that I had no idea! I had just started, and I really didn’t know how we managed to do such a good job with the (seemingly) small group. After working here for a while longer and thinking about how we do things compared with other teams I’ve worked for, I’ve come up with a few thoughts on why our small team is successful.

Simplicity: Office (on both platforms) is sometimes referred to as “bloated” or “complicated.” While we do have a very complex feature set, I think that our program managers (the people that design features) have done a great job of keeping the product simple and straightforward. There are many areas that have the potential to be needlessly complex, and we’ve managed to keep things smart. This simplicity is not only the result of good testing and design, but it also serves to help testers own more areas completely, and leads to better...

Knowledge: Each person on the team knows their product very well, as well as the other products in the Office suite. When you have dozens of people testing an application, no individual person is necessarily going to look outside of their areas to see the big picture. I think understanding how features relate to, and interact with, each other helps us deliver much more consistent applications, something that is key for Mac users to accept them. When you have too many people working to ensure the quality of something, you lose sight of the fact that you’re not just delivering 100 individual things, you’re delivering a whole product. Individual testers and not just leads/managers knowing more about the product leads to better...

Prioritization: Anyone who works in software knows that not every feature gets finished, and not every bug gets fixed. If we waited for that to happen, we would never release a product. The fact that we know what needs to work, how it should work, and when it needs to be done lets us make much better decisions about where our priorities lie. This kind of prioritization happens in any product, but usually at a higher level. As a part of a small team, I feel like I have the ability to affect real change, and make a difference by speaking up about our products’ quality. Empowerment of "the little guy" leads to the most important factor in our success in the MacBU...

Passion: The people I work with are committed to the team and to the products. Very few people end up working in the MacBU because it’s an easier job than the rest of Microsoft or the quickest way up the corporate ladder. People are drawn to this group because they care about the product, enjoy working on the platform, and are committed to it. There is no substitute for a someone enjoying their job, it makes you much more productive and improves the entire process. So I guess to sum up why I think the MacBU is as efficient as we are: Knowledge of what you need to do, and your personal ownership of it makes for a very committed team that is able to accomplish things that would normally take a much larger group of people.

I hope I don't come off like some kind of management training handbook or motivational speaker. At least I didn't arrange those into a cheesy acronym! The new Joe LeBlanc SKPP small team testing methodology!@# :rolleyes: I'm just making an outsider's first impression of how things work around here, and why I think we're as good as we are.

Have a great weekend!

Let's go way back

Schwieb — Tue, 26 Sep 2006 23:10:00 GMT

It's Schwieb again, this time with a look back deep into MacBU history. No, not any public history -- I'll talking about my own history in MacBU. Brad Post's blog entries about tracking down gnarly bugs made me remember a particular problem I worked on a very long time ago... I've got a lot of references in here to some pretty old OS technologies so for those of you newer or non-programmers out there, bear with me!

So, here's the scene: It is summer 1997. I've been at Microsoft for just under one year, and in the MacBU for about 7 months. We're well past code-complete and working feverishly to hammer out the bugs and ship Office 98. Beyond my few months in the MacBU, all my Mac experience was self-taught on my IIsi in college. I'd taken a course using 68k assembly and helped teach PPC assembly for a few semesters, but had worked my way through the New Inside Macintosh books myself. (Go search the net for UsenEdit or MacGrid, if you really want to embarrass me...)

There I am in my office, poking away at bugs in Mac Excel, when I get assigned this doozy. The steps are:

Position the Office folder window so it is off to the side away from the splash screen
Turn on Balloon Help
Double-click on Excel to launch it
Immediately after double-clicking, move your mouse a bit so it is over the Office folder window instead of over the Excel icon
BUG: Excel crashes right after showing its splash screen.

OK, that's weird... Excel 98 doesn’t actually do anything with Balloon help, especially not during boot. So, I first try to reproduce the bug. Sure enough, when I run through the scenario Excel crashes. The error message is "unimplemented instruction." I'm still trying to deny that the bug should exist and so I try it a few more times, all with the same results. I even try to change some of the steps -- I discover that if Balloon Help is off then there is no crash. Also, if I don't move the mouse in step 4 but leave it hovering over the app icon, then there is also no crash. Very weird!

For those Kon and Bal aficionados following along, care to guess what the problem is yet?

I myself had no clue. At this point I fired up Excel under the Microsoft remote debugger that we were using and tried to look and see what was going on. As my usual practice, I first just let Excel run until it crashed so that I could see what the CPU state was and where we were in the code. Lo and behold, when Excel crashed the Visual Studio debugger showed me complete and utter garbage! I had no stack crawl and no source matching the current address, and the PPC registers looked like they were full of nonsense. When I disassembled the code around the crashed address, none of it was even valid PPC assembly! Did we have a stack trasher somewhere that was corrupting a return address so that we jumped into random memory? I just had no idea.

Next I tried the bug under the MSVC debugger, but this time I stepped through Excel's boot sequence line by line. I saw that we crashed calling HideWindow on the Excel splash screen. Hmmm. The WindowPtr was completely valid, the stack looked ok before we called HideWindow, and yet we died a horrible death. Very very weird!

Anyone with an answer yet?

Ok, since I wasn't getting any good answers out of the source-level debugger, I decided to run Excel with MacsBug enabled. I switched to my 2nd Mac and tried the bug there. Whoa -- we didn’t crash! I try removing MacsBug on that machine and running Excel again. Still no crash! I go back to my primary Mac -- Excel crashes. So I install MacsBug on that Mac and try yet again.

Yep, crash. But things look very strange... MacsBug says we've crashed in 68k code -- in other words, we've crashed inside some code that is still 68k even on the PowerPC. But Excel is 100% fully PPC... Did we crash inside the OS somewhere? SC6 gives some gobbledygook, so I'm still thinking we might have a stack trasher. But why did we end up in 68k-land then? As I take a closer look, I see that MacsBug also says "unimplemented instruction" and the instruction pointer is pointing to an A-Trap. The A-Trap in question is ShowHide.

Anyone? Anyone?

Ummm, wait, you say... Earlier I said we were crashing during a call to HideWindow, and now MacsBug says ShowHide? So which is it? I have no idea -- my code shows us calling HideWindow and MacsBug says ShowHide... In either case, why does MacsBug think the ATrap is unimplemented? Both system APIs have been around forever! Just for the heck of it, I tell MacsBug to step over the instruction. It works! Huh? Ok, so I tell MacsBug to step a little more. It does, and eventually gets back to PPC code, and suddenly the stack looks right! The PPC assembly even reports a branch to HideWindow.

Ok, now I'm very confused. Two different APIs, two different execution engines, an unimplemented instruction that works... What is going on here...? I tell MacsBug to continue, and Excel actually boots up. No crash! Very very weird!

Bueller?

So I start playing. As I watch the boot sequence very closely I see something not reported in the original bug. During boot, shortly after Excel puts up its splash screen, the OS puts up a balloon help window near my mouse cursor, over the Finder window in the background. Hmmm -- there's the link to Balloon Help. The text is the usual "this window belongs to the Finder. It is disabled because the Finder is in the background" or whatever Balloon Help used to say all those years ago. I also note that the crash always happens on my PowerPC 9600/233 and never on my 8100/100. While I continue testing out various scenarios I discover that MacsBug is quite consistent. If I set a breakpoint in our code on HideWindow and then step over, I end up crashing on a 68k call to ShowHide, but I can always continue and complete Excel's boot. Since the PPC stack always looks OK, I'm pretty sure the problem is not a stack trasher, but instead may be a bad patch somewhere. But where?

I start digging through Excel and find that Excel has patched 8-10 of the core Window Manager routines, including HideWindow and ShowHide. But the patches all look OK and are all purely PowerPC -- there is indeed no 68k code anywhere in Excel. So I dig a little further, and find that the shared Office code has also patched much of the Window Manager, including the same two APIs. In fact, different parts of Office that came from different projects originally have both patched the Window Manager! That looks ugly! But still, all the Office patches are completely PowerPC-based, and should not be causing problems with 68k ATraps.

My lead suggested I contact Apple Developer Technical Support, to see if they could help shed some light on the subject.

Anyone in DTS then or now have a solution? :)

Some time went by, I worked on other bugs, and then I got an email from Apple DTS. The DTS person too suspected a rogue patch but from a different angle. He asked if we had any self-modifying patches. Hmm, I dunno. So I went and dug a bit, and found that *gasp* yes we did! One of the patch sets in Office did not want to allow itself to be called recursively. It prevented this by overwriting its entry point with a jump to the original patch vector upon entry to the Office patch. It then called MakeDataExecutable() to ensure that the code cache for that entry point was flushed.

Ahh, what's that you say? MakeDataExecutable only works on the PowerPC cache? Why yes, that is indeed the problem! The Office patch tried to restore the link to the original vector by inserting a jump instruction in the same architecture as the original vector. If the original vector was a 68k patch we'd overwrite our routine descriptor with a "jmp <addr>" instruction. This meant we had a valid 68k instruction there but an invalid routine descriptor. The problem is that MakeDataExecutable was only flushing the PPC code cache, not the 68k code cache that was maintained by the 68k emulator in the operating system. This also explained why I only saw the crash on one of my machines! The 9600/233 had the newer dynamic recompiling emulator that was actually caching the routine descriptors; the original 68k emulator on the 8100 was not -- it was always fetching the 68k instructions from main memory and would read the modified code. MacsBug allowed us to continue after the crash because it always flushed all the code caches upon entry into MacsBug, thus masking the problem.

However, just identifying the problem was not enough... We had to fix it! DTS discovered that the function call we needed (FlushCodeCacheRange) was not implemented on the PowerPC because it affected 68k code only and thus we couldn't call it. (The funny thing is that TechNote 1127, which addressed this very API, wasn't written until a few months after this bug.) DTS wrote up some glue code that we could use to flush the emulator's cache and *poof* the bug was fixed.

Astute readers will notice that I still haven't identified the link to Balloon Help. Some more back-and-forth with DTS revealed that Balloon Help was the component that was still in 68k mode and it had its own patches on the Window Manager. If Balloon Help wasn't turned on, then there were no 68k patches in the patch chain for these APIs and we didn’t do the corruption dance. The window positioning was important because if the Finder window was under the splash screen Balloon Help wouldn’t put up a balloon about it, and if the mouse was over the application icon there wouldn't be a balloon either. Balloon Help had to be turned on and the mouse had to be over the empty content region of a finder window that was still visible during the splash screen sequence in order to trigger the bug!

Whew. That was a gnarly one to track down, particularly because while I knew the two architectures individually, I did not yet know the Mixed Mode Manager very well. Apple helped us out a lot and I'm sure had a good laugh at the convoluted nature of our patch code.

Looking back on this bug and the technologies involved so many years later, I'm pretty impressed with how far the Mac OS has come. Carbon events, Cocoa, HIObjects, Intel -- it all adds up to a nice robust platform upon which we can build anything. I've been in the MacBU for 10 years now (my 10-yr anniversary is this coming Saturday.) I can't imagine being anywhere else!

(By the way, one of the very first things we did for Office 2001 was rip out all uses of the Trap Manager.)

Bugs, how I love them ...

ExCntx — Thu, 31 Aug 2006 02:16:00 GMT

I love hunting them down and killing them. Hunting down and killing the developer who caused the bug would be just as much fun if it weren’t illegal in every country in the world, that and programmers would probably be an extinct species if every programmer who had a fix a bug killed off the guy who created it ... and of course if they introduced the bug … well you get the picture, it's not pretty.

The thing is, if you haven’t debugged a bug in an emulator, you haven’t really had to debug. Oh, I know people say debugging hardware is tough and I’ve had to do that, but debugging software that emulates hardware or even virtualizes hardware is an art and you really don’t know the how hard hunting down a bug can be until you’ve gotten an emulation bug.

The first hard bug I ever had to find was back in my Apple days (think System 6 ... yeah I'm that old :-). Essentially if the user created a new folder on the desktop and renamed it you corrupted the desktop database which essentially wiped your hard drive clean. I didn’t track it down but figured out a way to reproduce it. The engineer who came and looked at it introduced me to one my favorite programs: Macsbug .

From then on it was assembly or nothing. For me starting at the bottom was the way to go; who needs source code. Understand the machine, the compiler and the rest is pretty easy … until I met an emulator. Don’t get me wrong, I know that developers all day deal with tough bugs. Code is filled with them, but hands down in my 16 years of developing software the emulator bugs are the toughest. In the six years I worked on Virtual PC and Virtual Game Station I would say on AVERAGE my bugs took over a week or more to fix. That’s PER bug, at least 40 hours of my time, not working on 5 or 6 bugs at once, focusing my entire attention at one issue. On Virtual PC 7, in the 16 months I worked on it I think I only fixed 30 some odd bugs.

The doozy of them all has to be the malformed TLB entry. For those who need a brush up, Virtual PC was a full blown PC motherboard emulator. We emulated a Triton-FX/BX chipset from the Pentium 3 chip down to the A20 interrupt. In between all of that was memory management, disk, networking and host of other integration features. In the memory management unit for our emulator we dealt with page faults and memory page mappings … all of this to properly emulate the MMU and let Windows (or other x86 based OS) believe it was running on real hardware. TLB stands for Translation Look-aside Buffer, and it is used in managing page faults. Most of the code that did this was written back in the early days of Virtual PC by Eric Traut for when we have the original PowerPC processors and G4 machines (circa 1998-1999).

Okay, so cutting to the chase, when the G5 processor was introduced we had to do a great deal of work to re-write our low-level emulator to deal with a number of issues brought up by the G5. What we didn’t know is that lurking deep in the guts of the memory management was a very subtle malformed TLB bug. So here we are, 14 months of development, we’re ready to ship with XP SP2 (we held up shipping to make sure we could ship the latest service pack) and wouldn’t you know, 20% of the time on DUAL G5 processors, when running through the OOBUE (Out of box user experience) on XP SP2 Professional, Windows would crash, but somehow manage to recover. With XP SP2 Home it was only 10% of the time and when we ran XP SP2 Professional with a stripped down OOBUE, we got the failure rate down to 5%.

As anyone who has ever developed software would say… SHIP IT! Okay, no that’s not really what they would say. Usually it’s something like this: “ $*&^!@%@!#$!@) !(!*@#&@!#*(“ and if they are from New Jersey the explicatives go on for quite a while. Needless to say we finally did ship VPC 7.0 and found a fix for the TLB bug. It only took me 9 weeks and 100s of hours trying to get the bug to reproduce though automation and manual means, as well as having the smartest emulator gurus at Microsoft looking over my shoulder at the source code and helping me debug the problem.

When I thought I had the fix, we ran all the lab machines through an automated test script overnight. Each machine ran through the OOBUE about 100 times, and we had maybe 20 machines going, turning our lab into a sauna. I came in the next more and found out that we had a 0% failure rate. I can’t even put into words the feeling I get when I solve these tough problems, anyone who has tackled a tough bug knows what I am talking about.

That was the thing I loved the most about working on Virtual PC, no matter how good you thought you were there was always a bug like that waiting to challenge you. In some ways that challenge is what I am going to miss the most too.