Larry Osterman's WebLog

Confessions of an Old Fogey
  • Larry Osterman's WebLog

    It was 20 years ago today...


    Nope, Sgt Pepper didn’t teach the band to play.

    20 years ago today, a kid fresh out of Carnegie-Mellon University showed up at the door of the 10700 Northup Way, ready to start his first day at a real job.

    What a long strange trip it’s been.

    Over the past 20 years, I’ve:

    ·         Worked on two different versions of MS-DOS (4.0, 4.1).

    ·         Worked on three different versions of Lan Manager (1.0, 1.5, 2.0)

    ·         Worked on five different releases of Windows NT (3.1, 3.5, XP (SP2), W2K3 (SP1), Longhorn)

    ·         Worked on four different releases of Exchange (4.0, 5.0, 5.5, and 2000)

    I’ve watched my co-workers move on to become senior VP’s.  I’ve watched my co-workers leave the company.

    I’ve seen the children of my co-workers grow up, go to college, marry, and have kids.

    I’ve watched the younger brother of my kids babysitter who I met at 12 years of age grow up, go to college and come to work at Microsoft in the office around the corner from mine (that one is REALLY weird btw).

    I’ve seen strategy’s come and go (Lan Manager as an OEM product, then retail, then integrated with the OS).

    I’ve watched three different paradigm shifts occur in the software industry, and most of a fourth.  The first one was the shift of real computing to “personal” computers.  The second was the GUI revolution, the third was the internet, and now we’re seeing a shift to smaller devices.  We’re still not done with that one.

    I’ve watched Microsoft change from a “small software startup in Seattle” to the 800 pound gorilla everyone hates.

    I’ve watched Microsoft grow from 650ish people to well over 50,000.

    I’ve watched our stock grow and shrink.  I’ve watched co-workers fortunes rise and fall.

    I’ve watched governments sue Microsoft.  I’ve watched Governments settle with Microsoft.  I’ve seen Microsoft win court battles.  I’ve seen Microsoft lose court battles.

    I’ve watched the internet bubble start, blossom, and explode.

    I’ve watched cellular phones go from brick-sized lumps to something close to the size of matchbooks.

    I’ve seen the computer on my desktop go from a 4.77MHz 8088 with 512K of RAM and a 10M hard disk to a 3.2GHz hyper-threaded Pentium 4 with 1G of RAM and an 80G hard disk.

    I’ve watched the idea of multimedia on the PC go from squeaky beeps from the speaker to 8-channel surround sound that would rival audiophile quality products.

    I’ve watched video on the PC go from 640x350 Black&White to 32bit color rendered in full 3d with millions of polygons.

    When I started at Microsoft, the computer that they gave me was a 4.77MHz PC/XT, with a 10 megabyte hard disk, and 512K of RAM.  I also had a Microsoft Softcard that increased the RAM to 640K, and it added a clock to the computer, too (they didn’t come with one by default)!  Last month, I bought a new computer for my home (my old one was getting painfully slow).  The new computer is a 3.6GHz Pentium 4, with 2 GIGABYTES(!) of RAM, and a 400 GIGABYTE hard disk.  My new computer cost significantly less than the first one did.  If you index for inflation, the new computer is at least an order of magnitude cheaper.

    I still have the letter that Microsoft sent me confirming my job offer.  It’s dated January 16th, 1984.  It’s formatted in Courier, and the salary and stock option information is written in ink.  It’s signed (in ink) by Steve Ballmer.  The offer letter also specifies the other benefits; it’s not important what they are.  I also have Steve’s business card – his job title?  VP, Corporate Staffs.  Yup, he was head of HR back then (he did lots of other things, but that’s what his title was).  I also have the employee list they gave out for the new hires, as I said before; there are only about 600 people on it.  Of those 600 people, 48 of them are still with Microsoft.  Their job titles range from Executive Assistant, to UK Project Troubleshooter, to Architect, to Director. 

    The only person who I interviewed with when I started is still at Microsoft, Mark Zbikowski.  Mark also has the most seniority of anyone still at Microsoft (except for Steve Ballmer and Bill Gates).

    When I started in the Lan Manager group, Brian Valentine was a new hire.  He was a test lead in the Lan Manager group, having just joined the company from Intel.  He (and Paul Maritz) used to tell us war stories about their time at Intel (I particularly remember the ones about the clean desk patrol).

    In the past twenty years, I’ve had 16 different managers:  Alan Whitney (MS-DOS 4.0); Anthony Short (MS-DOS 4.0); Eric Evans (MS-DOS 4.0, MS-DOS 4.1); Barry Shaw (Lan Manager 1.0); Ken Masden (Lan Manager 1.5, Lan Manager 2.0); Dave Thompson (Lan Manager 2.0, Windows NT 3.1); Chuck Lenzmeier (Windows NT 3.5); Mike Beckerman (Tiger); Rick Rashid (Tiger); Max Benson (Exchange 4.0, 5.0, 5.5); Soner Terek (Exchange 5.5, Exchange 2000); Jon Avner (Exchange 2000); Harry Pyle (SCP); Frank Yerrace (Longhorn); Annette Crowley (Longhorn) and Noel Cross (Longhorn).

    I’ve moved my office 18 different times (the shortest time I’ve spent in an office: 3 weeks).  I’ve lived through countless re-orgs.  On the other hand, I’ve never had a reorg that affected my day-to-day job.

    There have been so many memorable co-workers I’ve known over the years.  I can’t name them all (and I know that I’ve missed some really, really important ones), but I’ll try to hit some highlights.  If you think you should be on the list but aren’t, blame my poor memory, I apologize, and drop me a line!

    Gordon Letwin – Gordon was the OS guru at Microsoft when I started, he was the author of the original H19 terminal ROM before coming to Microsoft.  In many ways, Gordon was my mentor during my early years at Microsoft.

    Ross Garmoe – Ross was the person who truly taught me how to be a software engineer.  His dedication to quality continues to inspire me.  Ross also ran the “Lost Lambs” Thanksgiving Dinner – all of us in Seattle without families were welcome at Ross’s house where his wife Rose and their gaggle of kids always made us feel like we were home.  Ross, if you’re reading this, drop me a line :)

    Danny Glasser – Danny had the office across the hall from me when I was working on DOS Lan Manager.  He’s the guy who gave me the nickname of “DOS Vader”.

    Dave Cutler – Another inspiration.  He has forgotten more about operating systems than I’ll ever know.

    David Thompson – Dave was the singularly most effective manager I’ve ever had.  He was also my least favorite.  He pushed me harder than I’ve ever been pushed before, and taught me more about how to work on large projects than anyone had done before.  Valorie was very happy when I stopped working for him.

    David Weise – David came to Microsoft from Dynamical Systems Research, which I believe was Microsoft’s third acquisition.  He owned the memory management infrastructure for Windows 3.0.

    Aaron Reynolds – Author of the MS-NET redirector, one of the principal DOS developers.

    Ralph Lipe –Ralph designed most (if not all) of the VxD architecture that continued through Win9x. 

    David, Aaron, and Ralph formed the core of the Windows 3.0 team; it wouldn’t have been successful without them.  Collectively they’re the three people that I believe are most responsible for the unbelievable success of Windows 3.0.  Aaron retired a couple of years ago; David and Ralph are still here.  I remember David showing me around building 3 showing off the stuff in Windows 3.0.  The only thing that was going through my mind was “SteveB’s going to freak when he sees this stuff – this will blow OS/2 completely out of the water”.

    Paul Butzi – Paul took me for my lunch interview when I interviewed at Microsoft.  He also was in the office next to mine when I started (ok, I was in a lounge, he was in an office).  When I showed up in a suit, he looked at me and started gagging – “You’re wearing a ne-ne-ne-neckt….”  He never did get the word out.

    Speaking of Paul.  There was also the rest of the Xenix team:  Paul Butzi, Dave Perlin, Lee Smith, Eric Chin, Wayne Chapeski, David Byrne, Mark Bebie (RIP), Neil Friedman and many others.  Xenix 386 was the first operating system for the Intel 386 computer (made by Compaq!).  Paul had a prototype in his office, he had a desk fan blowing on it constantly, and kept a can of canned air handy in case it overheated.

    Ken Masden – the man who brought unicycle juggling to Microsoft.

    All of the “core 12”: Dave Cutler (KE), Lou Perazzoli (MM), Mark Lucovsky (Win32), Steve Wood (Win32, OB), Darryl Havens (IO), Chuck Lenzmeier (Net), John Balciunas (Bizdev), Rob Short (Hardware), Gary Kimura (FS), Tom Miller (FS),  Ted Kummert (Hardware), Jim Kelly (SE), Helen Custers (Inside Windows NT), and others.  These folks came to Microsoft from Digital Equipment with a vision to create something brand new.  As Tom Miller put it, it was likely to be the last operating system ever built from scratch (and no, Linux doesn’t count – NT was 100% new code (ok, the command interpreter came from OS/2), the Linux kernel is 100% new, but the rest of the system isn’t).  And these guys delivered.  It took longer than anyone had originally planned, but they delivered.  And these guys collectively taught Microsoft a lesson in how to write a REAL operation system, not a toy operating system like we’d been working on before.  Some day I’ll write about Gary Kimura’s coding style.

    Brian Valentine – Brian is without a doubt the most inspirational leader at Microsoft.  His ability to motivate teams through dark times is legendary.  I joined the Exchange team in 1994, the team was the laughing stock at Microsoft for our inability to ship product (Exchange had been in development for almost six years at that point), and we still had another year to go.  Brian led the team throughout this period with his unflagging optimism and in-your-face, just do it attitude.  For those reading this on the NT team: The Weekly World News was the official newspaper of the Exchange team LONG before it was the official newspaper of the Windows team.

    Max Benson – Max was my first manager in Exchange.  He took a wild chance on a potentially burned out engineer (my time in Research was rough) and together we made it work.

    Jaya Matthew – Jaya was the second person I ever had report to me; her pragmatism and talent were wonderful to work with.  She’s also a very good friend.

    Jim Lane, Greg Cox, and Ardis Jakubaitis – Jim, Greg, Ardis, Valorie and I used to play Runequest together weekly.  When I started, they were the old hands at Microsoft, and their perspectives on the internals of the company were invaluable.  They were also very good friends.

    And my list of co-workers would not be complete without one other:  Valorie Holden.  Yes, Valorie was a co-worker.  She started at Microsoft in 1985 as a summer intern working on testing Word and Windows 1.0.  While she was out here, she accepted my marriage proposal, and we set a date in 1987.  She went back to school, finished her degree, and we got married.  After coming out here, she started back working at Microsoft, first as the bug coordinator for OS/2, then as Nathan Myhrvold’s administrative assistant, then as a test lead in the Windows Printing Division, eventually as a program manager over in WPD.  Valorie has stood by my side through my 20 years at Microsoft; I’d never have made it without her unflagging support and advice (ok, the threats to my managers didn’t hurt either).

    There’ve been good times: Getting the first connection from the NT redirector to the NT server; Shipping Exchange 4.0; Shipping Exchange 2000 RC2 (the ship party in the rain).  Business trips to England.  Getting a set of cap guns from Brian Valentine in recognition of the time I spent in Austin for Lan Manager 2.0 (I spent 6 months spending Sunday-Wednesday in Austin, Thursday-Saturday in Redmond).

    There’ve been bad times:  Reorgs that never seemed to end.  Spending four years in ship mode (we were going to ship “6 months from now” during that time) for NT 3.1 (Read Showstopper! for more details).  The browser checker (it took almost ten years to get over that one).  A job decision decided over a coin toss.  Working in Research (I’m NOT cut out to work in research).

    But you know, the good times have far outweighed the bad, it’s been a blast.  My only question is: What’s in store for the next twenty years?


    Edit: Forgot some managers in the list :)

    Edit2: Missed Wayne from the Xenix team, there are probably others I also forgot.

    Edit3: Got some more Xenix developers :)


  • Larry Osterman's WebLog

    Should I check the parameters to my function?


    I just had an interesting discussion with one of the testers in my group.

    He had just finished filing a series of bugs against our components because they weren’t failing when he passed bogus pointers to the API.  Instead, they raised a 0xC0000005 exception and crashed his application.

    The APIs did fail if he passed a null pointer in, with E_POINTER. 

    But he felt that the API should check all the bogus pointers passed in and fail with E_POINTER if the pointer passed in didn’t point to valid memory.

    This has been a subject of a lot of ongoing discussion over the years internally here at Microsoft.  There are two schools of thought:

    School one says “We shouldn’t crash the application on bogus data.  Crashing is bad.  So we should check our parameters and return error codes if they’re bogus”.

    School two says “GIGO – if the app hands us garbage, then big deal if it crashes”.

    I’m firmly in the second camp (not surprisingly, if you know me).  There are a lot of reasons for this.  The biggest one is security.  The way you check for bad pointers on Win32 is by calling the IsBadReadPtr and IsBadWritePtr API.  Michael Howard calls these APIs “CrashMyApplication” and “CorruptMemoryAndCrashMySystem” respectively.  The problem with IsBadReadPtr/IsBadWritePtr is that they do exactly what they’re advertised as doing:  They read and/or write to the memory location specified, with an exception handler wrapped around the read/write.  If an exception is thrown, they fail, if not, they succeed.

    There are two problems with this.  The only thing that IsBadReadPtr/IsBadWritePtr verifies is that at the instant that the API is called, there was valid memory at that location.  There’s nothing to prevent another thread in the application from unmapping the virtual address passed into IsBadReadPtr immediately after the call is made.  Which means that any error checks you made based on the results of this API aren’t valid (this is called out in the documentation for IsBadWritePtr/IsBadReadPtr).

    The other one is worse.  What happens if the memory address passed into IsBadReadPtr is a stack guard page (a guard page is a page kept at the bottom of the stack – when the system top level exception handler sees a fault on a guard page, it will grow the threads stack (up to the threads stack limit))?  Well, the IsBadReadPtr will catch the guard page exception and will handle it (because IsBadReadPtr handles all exceptions).  So the system exception handler doesn’t see the exception.  Which means that when that thread later runs, its stack won’t grow past the current limit.  By calling IsBadReadPtr in your API, you’ve turned an easily identifiable application bug into a really subtle stack overflow bug that may not be encountered for many minutes (or hours) later.

    The other problem with aggressively checking for bad parameters on an API is that what happens if the app doesn’t check the return code from the API?  This means that they could easily have a bug in their code that passes a bogus pointer into IsBadWritePtr, thus corrupting memory.  But, since they didn’t check the return code, they don’t know about their bug.  And, again, much later the heap corruption bug that’s caused by the call to IsBadWritePtr shows up.  If the API had crashed, then they’d find the problem right away.

    Now, having said all this, if you go with school two, you’ve still got a problem – you can’t trust the user’s buffers.  At all.  This means you’ve got to be careful when touching those buffers to ensure that you’re not going to deadlock the process by (for instance holding onto a critical section while writing to the user’s buffer).

    The other thing to keep in mind is that there are some situations where it’s NOT a good idea to crash the user’s app.  For example, if you’re using RPC, then RPC uses structured exception handling to communicate RPC errors back to the application (as opposed to API return codes).  So sometimes you have no choice but to catch the exceptions and return them.  The other case is if someone has written and shipped an existing API that uses IsBadReadPtr to check for bad pointers on input, it may not be possible to remove this because there may be applications that depend on this behavior.

    So in general, it’s a bad idea to use IsBadXxxPtr on your input parameters to check for correctness.  Your users may curse you for crashing their app when they screw up, but in the long term, it’s a better idea.

  • Larry Osterman's WebLog

    Farewell to one of the great ones

    Yesterday was the last day at Microsoft for David WeiseI've written about David (in passing) in the past, but never in detail.

    David started at Microsoft in 1986, when Microsoft acquired Dynamical Systems Research.  Before founding DSR, he was a member of the ORIGINAL MIT blackjack team - not the latecomers that you see in all the movies, but the original team, back in the 1970s.  According to Daniel Weise (David's twin brother), they ran it like an investment company - MIT people could invest money in the blackjack team, and the blackjack team would divide their winnings up among them.  Apparently RMS was one of the original investors, during David's going away party, Daniel joked that the FSF was founded on David's blackjack winnings :)

    After leaving Princeton with a PhD in molecular biophysics, David, Chuck Whitmer, Nathan and Cameron Myhrvold, and a few others founded DSR to create a "Topview" clone.  For those new to the industry, Topview was a text based multitasking shell that IBM created that was going to totally revolutionize the PC industry - it would wrest control of the platform from Microsoft and allow IBM to maintain its rightful place as leader of the PC industry.  Unfortunately for IBM, it was an utter flop.

    And, as Daniel pointed out, it was unfortunate for DSR.  Even though their product was twice as fast as IBMs and 2/3rds the size, when you base your business model on being a clone of a flop, you've got a problem.

    Fortunately, at the time, Microsoft was also worried about Topview, and they were looking for a company that understood the Topview environment so that if it was successful, Microsoft would have the ability to integrate Topview support into Windows.

    Finding DSR may have been one of the best acquisitions that Microsoft ever made.  Not only did they find the future CTO (and founder of Microsoft Research) Nathan Myhrvold, but they also hired David Weise.

    You see, the DSR guys were wizards, and David was a wizard's wizard.  He looks at programs and makes them smaller and faster.  It's absolutely magical to watch him at his work.

    I (and others) believe that David is single handedly responsible for making Microsoft over a billion dollars.  He's also (IMHO) the person who is most responsible for the success of Windows 3.0.

    Everywhere David worked, he dramatically improved the quality of the product.  He worked on the OS/2 graphics drivers and they got faster and smarter.  He (and Chuck) figured out tricks that even the designers of the hardware didn't realize could be done.

    And eventually, David found himself in the Windows group with Aaron Reynolds, and Ralph Lipe (and several others).

    Davids job was to move the graphics drivers in windows into protected mode on 286 and better processors (to free up precious memory below 640K for Windows applications).  He (and Chuck) had already figured out how to get normal Windows applications to use expanded memory for their code and data, but now he was tackling a harder  problem - the protected mode environment is subtler than expanded memory - if you touched memory that wasn't yours, you'd crash.

    David succeeded (of course).  But David, being David, didn't stop with the graphics drivers.

    He (along with Murray Sargent, creator of the SST debugger) also figured out how to get normal Windows applications running in protected mode.

    Which totally and utterly and irrevocably blew apart the 640K memory barrier.

    I remember wandering over to the Windows group over in Building 3 to talk to Aaron Reynolds about something to do with the MS-DOS redirector (I was working on DOS Lan Manager at the time).  I ran into David, and he called me into his office "Hey, look at what I've got working!".

    He showed me existing windows apps running in protected mode on the 286.  UNMODIFIED Windows 1.0 applications running in protected mode.

    He then ran me around the rest of the group, and they showed me the other stuff they were working on.  Ralph had written a new driver architecture called VxD.  Aaron had done something astonishing (I'm not sure what).  They had display drivers that could display 256 color bitmaps on the screen (the best OS/2 could do at the time was 16 colors).

    My jaw was dropping lower and lower as I moved from office to office.  "Oh my goodness, you can't let Steve see this, he's going to pitch a fit" (those aren't quite the words I used, but this is a family blog).

    You see, at this time, Microsoft's systems division was 100% focused on OS/2 1.1.  All of the efforts of the systems division were totally invested in OS/2 development.  We had invested literally tens of millions of dollars on OS/2, because we knew that it was the future for Microsoft.  OS/2 at the time just ran a single DOS application at a time, and it had only just recently gotten a GUI (in 1989).  It didn't have support for many printers (only about 5, all made by IBM, and (I believe) the HP Laserjet).

    And here was this little skunkworks project in building three that was sitting on what was clearly the most explosive product Microsoft had ever produced.  It was blindingly obvious, even at that early date - Windows 3.0 ran multiple DOS applications in virtual x86 machines.  It ran Windows applications in protected mode, breaking the 640K memory barrier.  It had a device driver model that allowed for development of true 32bit device drivers.  It supported modern displays with color depths greater than had been available on PC operating systems. 

    There was just no comparison between the two platforms - if they had to compete head-to-head, Windows 3.0 would win hands down.

    Btw, David had discussed it with Steve (I just learned that yesterday).  As David put it, he realized that this was potentially an issue, so he went to Steve, and told him about it.  Steve asked Dave to let him know when he'd made progress.  That night, David was up until 5AM working on the code, he got it working, and he'd left it running on his machine.  He left a note on SteveB's office door saying that he should stop by David's office.  When David got in the next day (at around 8AM), he saw that his machine had crashed, so he knew that Steve had come by and seen it.

    He went to Steve's office, and they had a chat.  Steve's only comment was that David should tell his manager and his manager's manager so that they'd not be surprised at the product review that was going to happen later that day.  At the product review, Steve and Bill greenlighted the Windows 3.0 project, and the rest was history.  My tour was apparently a couple of days after that - it was finally ok to let people know what the Windows 3.0 team was doing.

    The rest was history.  At its release, Windows 3.0 was the most successful software project in history, selling more than 10 million copies a month, and it's directly responsible for Microsoft being where it is today.

    And, as I mentioned above, David is responsible for most of that success - if Windows 3.0 hadn't run Windows apps in protected mode, then it wouldn't have been the unmitigated success it was.

    David's spent the last several years working in linguistics - speech generation, etc.  He was made a distinguished engineer back in 2002, in recognition of his contribution to the industry. The title of Distinguished Engineer is the title to which all Microsoft developers aspire, it is literally the pinnacle of a developers career at Microsoft when they're named DE - other DE's include Dave Cutler, Butler Lampson, Jim Gray, Anders Hejlsberg.  This is unbelievably rarified company - these are the people who have literally changed the world.

    And David absolutely belongs in their company.

    David's leaving to learn more about the state of molecular biology today, he wants to finally be able to use his PhD, the field has changed so much since he left it, and it's amazing what's happening in it these days.

    As I said as I was leaving his goodbye party:

    "Congratulations, good luck, and, from the bottom of my heart, thank you".

    Bonne Chance David, I wish you all the best.  When you get your Nobel Prize, I'll be able to say "I knew him back when he worked at Microsoft".


    Edit: Corrected David's PhD info based on Peter Woit's blog post here.  Sorry David, and thanks Peter.

    Edit2: Grey->Gray :)  Thanks Jay

  • Larry Osterman's WebLog

    What is localization anyway?

    I may be stomping on Michael Kaplan's toes with this one, but...

    I was reading the February 2005 issue of Dr. Dobbs Journal this morning and I ran into the article "Automating Localization" by Hew Wolff (you may have to subscribe to get access to the article).

    When I was reading the article, I was struck by the following comment:

     I didn't think we could, because the localization process is prety straightforward. By "localization", I mean the same thing as "globalization" (oddly) or "internationalization." You go through the files looking for English text strings, and pull them into a big "language table," assigning each one a unique key

    The first thing I thought was what an utterly wrong statement.  The author of the article is conflating five different concepts and calling them the same thing.  The five concepts are: localizability, translation, localization, internationalization, and globalization.

    What Hew's talking about is "localizability" - the process of making the product localizable.

    Given that caveat, he's totally right in his definition of localizability - localizability is the process of extracting all the language-dependant strings in your binary and putting them in a separate location that can be later modified by a translator.

    But he totally missed the boat on the rest of the concepts.

    The first three (localizability, translation, and localization) are about resources:

    • Localizability is about enabling translation and localization.  It's about ensuring that a translator has the ability to modify your application to work in a new country without recompiling your binary.
    • Translation is about converting the words in his big "language table" from one language to another.  Researchers love this one because they think that they can automate this process (see Google's language tools as an example of this).
    • Localization is the next step past translation.  As Yoshihiko Sakurai mentioned to Michael in a related discussion this morning "[localization] is a step past translation, taking the certain communication code associated with a certain culture.  There are so many aspects you have to think about such as their moral values, working styles, social structures, etc... in order to get desired (or non-desired) outputs.  This is one of the big reasons that automated translation tools leave so much to be desired - humans know about the cultural issues involved in a language, computers don't.

    Internationalization is about code.  It's about ensuring that the code in your application can handle strings in a language sensitive manner.  Michael's blog is FULL of examples of internationalization.  Michael's article about Tamil numbers, or deeptanshuv's article about the four versions of "I" in Turkish are great examples of this.  Another example is respecting the date and time format of the user - even though users in the US and the UK both speak English (I know that the Brits reading this take issue with the concept of Americans speaking English, but bear with me here), they use different date formats.  Today is 26/01/2005 in Great Britain, but it's 01/26/2005 here in the US.  If your application displays dates, it should automatically adjust them.

    Globalization is about politics.  It's about ensuring that your application doesn't step on the policies of a country - So you don't ever highlight national borders in your graphics, because you might upset your customers living on one side or another of a disputed border. I do want to be clear that this isn't the traditional use of globalization, maybe a better word would be geopoliticization, but that's too many letters to type, even for me, and since globalization was almost always used as a synonym for internationalization, I figured it wouldn't mind being coopted in this manner :)

    Having said that, his article is an interesting discussion about localization and the process of localization.  I think that the process he went through was a fascinating one, with some merit.  But that one phrase REALLY stuck in my craw.

    Edit: Fixed incorrect reference to UK dates - I should have checked first :)  Oh, and it's 2005, not 2004.

    Edit2: Added Sakurai-san's name.

    Edit3: Added comment about the term "globalization"

  • Larry Osterman's WebLog

    Why does Windows share the root of your drive?


    Out-of-the box, a Windows system automatically shares the root of every hard drive on the machine as <drive>$ (so you get C$, D$, A$, etc).

    The shares are ACL'ed so that only members of the local administrative group can access them, and they're hidden from the normal enumeration UI (they're included in the enumeration APIs but not in the UI (as are all shares with a trailing $ in their name).

    One question that came up yesterday was why Windows does this in the first place.

    The answer is steeped in history.  It goes way back to the days of Lan Manager 1.0, and is a great example of how using your own dogfood helps create better products.

    Lan Manager was Microsoft's first attempt at competing directly with Novell in networking.  Up until that point, Microsoft produced an OEM-only networking product called MS-NET (I have a copy of the OEM adaptation kit for MS-NET 1.1 in my office - it was the first product I ever shipped at Microsoft).

    But Lan Manager was intended as a full solution.  It had a full complement of APIs to support administration, supported centralized authentication, etc.

    One of the key features for Lan Manager was, of course, remote administration.  The server admin could sit in their office and perform any administrative tasks they wanted to on the computer.

    This worked great - the product was totally living up to our expectations...

    Until the day that the development lead for Lan Manager (Russ (Ralph) Ryan) needed to change a config file on the LanMan server that hosted the source code for the Lan Manager product.  And he realized that none of the file shares on the machine allowed access to the root directory of the server!  He couldn't add a new share remotely, because the UI for adding file shares required that you navigate through a tree view of the disk - and since the root wasn't shared, he could only add shares that lived under the directories that were already shared.

    So he had to trudge from his office to the lab and make the config change to the server.

    And thus a new feature was born - by default, Lan Manager (and all MS networking products to this day) shares the root of the drives automatically to ensure that remote administrators have the ability to access the entire drive.   And we'd probably have never noticed it unless we were dogfooding our products.

    Nowadays, with RDP and other more enhanced remote administration tools, it's less critical, but there are a boatload of products that rely on the feature.

    Note1: You can disable the automatic creation of these shares by going to this KB article.

    Note2: The test lead for the Lan Manager product was a new hire, fresh from working at Intel who went by the name of Henry (Brian) Valentine.

  • Larry Osterman's WebLog

    Beep Beep

    What's the deal with the Beep() API anyway?

    It's one of the oldest Windows API, dating back to Windows 1.0.  It's also one of the few audio APIs that my team doesn't own.  The Beep API actually has its own dedicated driver (beep.sys).  The reason for this is that the Beep() API works totally differently from any other audio API in the system.

    Back when IBM built the first IBM PCs, they realized that they needed to have the ability to do SOME level of audio, even if it wasn't particularly high quality.  So they built a speaker into the original PC hardware.

    But how do you drive the speaker?  It turns out that the original PC hardware used an 8253 programmable interval timer to control the system hardware timer.  The 8253 was a pretty cool little chip - it would operate in 5 different modes - one shot timer, interrupt on terminal count, rate generator, square wave generator, software strobe or hardware strobe.  It also contained three independent counters - counter 0 was used by the operating system, counter 1 was reserved for the hardware.  The third counter, counter 2 was special.  The IBM hardware engineers tied the OUT2 line from the 8253 to the speaker line, and they programmed the timer to operate in square wave generation mode.

    What that means is that whenever the 2nd counter of the 8253 counted to 0, it would toggle the output of the OUT2 line from the 8253.  This gave the PC a primitive way of generating very simple tones.

    The original Windows Beep() API simply fiddled the controls on the 8253 to cause it to generate a square wave with the appropriate frequency, and that's what Beep.sys continues to do.  Legacy APIs can be hard to remove sometimes :)

    Nowadays, the internal PC speaker is often also connected to the PCs audio solution, that allows the PC to have sound even when there are no external speakers connected to the machine.

    In addition to the simple beep, some very clever people figured out how they could use the 8253 to generate honest to goodness audio, I'm not sure how they succeeded in doing it but I remember someone had a PC speaker based sound driver for DOS available at one point - it totally killed your PCs performance but it DID play something better than BEEEEEEP.

    Edit: s/interrupt conroller/interval timer/

    Edit2: fixed description of channel 1 (in case someone comes along later and decides to depend on my error).

  • Larry Osterman's WebLog

    How did we make the DOS redirector take up only 256 bytes of memory?

    In one of my early posts, I mentioned a status review we had with BillG for the DOS Lan Manager redirector (network filesystem).

    I also talked to Robert Scoble about this in the last of my Channel9 videos.  One thing that somehow got missed in both the original article (later updated) and the video was our reaction to Bill's feedback.

    The simple answer is that we fixed the problem.  My team didn't do much with the transports and network drivers (because they were out of our scope), but we were able to do something about the footprint of the redir.exe program (it was a T&SR application).

    When we were done with it, I managed to shrink the below 640K running size of redirector to 128 bytes in size, beyond which I couldn't figure out how to go.

    The question that obviously comes up is: How did you manage to do that?  Raymond, please forgive me for what I'm about to disclose, for within this tale lie dragons.  This discussion is for historical purposes ONLY.  I don't recommend it as a practice.

    The MS-DOS redirector was actually originally written as a part of the MSDOS.SYS (IBMDOS.COM) binary.  For obvious reasons (not every user in the world had a network card, especially in 1984), the redirector was split out from the DOS binary after the product shipped.  In fact, when I took over the redirector project, the binary used to link with hundreds of unresolved external symbol errors (because the redirector linked with some, but not all of the MS-DOS binaries).  One of the first things that I did while working on the project was to clean this up so that the redirector would cleanly link without relying on the MS-DOS objects.  But being a part of MS-DOS, it was written in "tiny" mode - the code and data were commingled internally.

    The first thing I did when trying to shrink the footprint of the redirector was to separate the code and data segments.  Today, this seems utterly obvious, but in 1986, it was a relatively radical idea, especially for real-mode software.  Once I had split the data and code, I was able to make the data segment relocatable.  This change was critical, because it enabled us to do a boatload of things to reduce our footprint.  One thing to keep in mind about the redirector was that even though the data (and eventually code) was relocatable, the motion wasn't dynamic.

    The first thing I did was to lay the redirector's code and data as follows:


    Initialization Code


    Initialization Data/Dynamic data (allocated after startup)

    By laying out the code and data this way, I could slide the data over the initialization code after the initialization was done.  It wasn't that much of a real savings, however, since the original redirector simply started the dynamic data at the start of the initialization code (and left it uninitialized).

    The next thing to do was to take advantage of a quirk in the 286 processor.  The 8086 could only address one megabyte of memory, all the memory above 640K was reserved for system ROMs. (A quick aside: DOS could (and did) take advantage of more than 640K of RAM - DOS could address up to 1M of RAM, all the processor could support if it wasn't for the system ROMs.  In particular, there were several 3rd party memory cards that allowed mapping memory between 640K and 0xB0000, which was the start of video memory).  With the addition of the 286 processor, the machine could finally address more than 1M of RAM.  It turns out that if the machine had more than 640K of RAM, most systems mapped the memory above 640K to above 1M.  Unfortunately, there were a number ofapplications that depended on the fact that the 8086 could only address 1M of RAM, and performed arithmetic that assumed that physical address 0xFFFF0+0x30=0x000020.  To control this, the PC/AT and its successors defined a software controllable pin called the "A20 line" - if it was disabled , memory access between 1M and 1M+64K was redirected to 0, if it was enabled , then it was mapped to real memory.  This is really complicated, but the effect was that if you enabled the A20 line, an application could have access to 64K of additional memory that didn't impact any running MS-DOS applications!  This 64K was known as the "High Memory Area", or HMA.

    Because the powers that be knew that this would be a highly contentious piece of real estate (everyone would want to party on it), Microsoft (or Intel, or IBM, I'm not sure who) wrote a specification and a driver called HIMEM.SYS.  HIMEM.SYS's purpose was to arbitrate access to that 64K chunk of RAM.

    Well, for the DOS Lanman redirector, we wanted to use that area, so if we were able to reserve the region via himem.sys, we moved the data (both dynamic and static) up to that memory.  On every entry to the redirector, we enabled the A20 line (via himem.sys), and on every exit, we disabled the A20 line.

    That saved about 30K of the 60K MS-DOS footprint, so far so good. 

    The next step in the process was to remove our dependencies on himem.sys.  Around this time, Lotus, Intel and Microsoft had defined a specification for an expanded memory manager, known as LIM.  This allowed a 3rd party memory card to bank swap memory into the 0xA0000->0xFFFFF memory region.  Marlin Eller joined the team about that time, and he wrote the code to move the data segment for the DOS redirector into LIM (if himem.sys wasn't available, and LIM was).  After finishing that work, he moved on to other projects within Microsoft.  That's where things stood for Lan Manager 1.5, the data had been removed, but nothing else.  A HUGE improvement, but we weren't satisfied.

    So far, we were just moving the data around, we hadn't done anything to deal with the 30K or so of code.

    The next thing we did was to split the redirector up still further:

    "Low" code
    "Low" data


    Initialization Code


    Initialization Data/Dynamic data (allocated after startup)

    We added a low code and data segment.  The "low" code segment contained all the external hooks into the redirector (interrupt handlers, etc), and code to enable the HMA and LIM segments.  We then moved the data into LIM memory, and the code into the HMA.  This was a bit trickier, but we managed.

    So we now had a low code segment that was about 2K or so, and the code and data was moved up out of the 640K boundary.  Normally, I'd be satisfied with this, but I love a challenge.

    The next step was to look long and hard at the low code.  It turns out that most of the low code didn't really NEED to be low, it was just convenient.  Since the code had been moved into the HMA, all I needed to do was to have a low-memory stub with enough code to enable the HMA, and dispatch to the corresponding function in high memory.

    The other thing I realized was that the MS-DOS PSP (Program Segment Prefix, the equivalent of a task in MS-DOS) contained 128 bytes of OS stuff, and 128 bytes of command line (this is where Raymond starts cringing).  Since the redirector didn't use the command line, I figured I could re-use that 128 bytes of memory for my stub to enable the high memory area.  And that's what I did - I used the 128ish bytes of command line to hold the interrupt dispatch routines for all the entrypoints to the redirector (there were about 4 of them), and pointers to the corresponding routines in the high memory area, and the code to enable (and disable) the HMA.

    And voila, I had a 0 footprint redirector.  The only negative that came from this was that applications that enumerated the "running" processes didn't handle the "code-in-the-command-line" thing.

    Btw, the work I did here was pretty much totally clean.  I used the linker to define the segments that were relocated, I didn't do any of the other sleazy things that MS-DOS programmers did to make their code small (like combining multiple instructions together relying on the relative offset of a jump instruction to form the first byte of a different instruction).  It was actually a pretty cool piece of work.

    Oh, and this description doesn't really give the full flavor of what had to be done to get this to work.  A simple example: Because I had to handle moving the data over the code that was performing the move - that meant that I need to first move the initialization code out of the way (past the end of the data), jump to the moved initialization code, move the data over the original initialization code, then terminate the application.

    But we eventually (for Lan Manager 2.2) had a 0 footprint redirector.  It took some time, and it didn't work for every configuration, but we DID make it work.


  • Larry Osterman's WebLog

    Open Source and Hot Rods


    I was surfing the web the other day and ran into someone linking to this article by Jack Lanier from Edmunds (the automotive newsletter people).

    The article's entitled "Friends Don't Let Friends Modify Cars".

    From the article:

    Today, it's difficult to make cars better and extremely easy to make them worse. Or dangerous.

    As a journalist driving modified cars, I've been sprayed with gasoline, boiling coolant, super-heated transmission fluid and nitrous oxide. (The latter was more entertaining than the former.) Several have burst into flames. Throttles have stuck wide open, brake calipers snapped clean off, suspensions ripped from their mounts and seatbelt mounting hardware has dropped into my lap. All this is on top of the expected thrown connecting rod, blown head gasket, exploded clutch, disintegrated turbocharger and broken timing belt.

    The vast majority of these vehicles were built by professionals. Many were from big-name tuners. Most performed as if they were constructed in a shop class at a high school with a lax drug policy. Once, after a suspension component fell off a car from a big-name tuner, the car actually handled better.

    For every modified and tuner car that performed better than stock, I've driven numerous examples that were slower. If they were quicker, it was often in an area that can't be used on the street. What's the use of gaining 0.2 second in the quarter-mile if the car is slower 0-60 mph? And costs $10,000 more?


    Recently, I autocrossed a pair of Subaru WRXs. One was a dead-stock WRX. The other, a tricked-out STi lowered with stiffer springs, shocks and bars and an exhaust kit and air filter. The STi is supposed to have an advantage of some 70 horsepower. Maybe the exhaust and filter moved the power up in the rev band where it couldn't be used. The lowered, stiffened STi regularly bottomed against its bump stops. When a car hits its bump stops, the spring rate goes to infinity and tire grip drops to near zero. This caused the STi to leap into the worst understeer I've experienced with inflated front tires. Meanwhile, in the unmodified WRX, I could be hard in the throttle at the same point. The result: The dead-stock WRX was at least as quick as the STi and far easier to drive. Easy to make worse, harder to make better

    I read this article and was struck by the similarities between this and the open source vs COTS model.

    COTS (Commercial Off The Shelf) software is equivalent to a stock automobile.  They're built by professional engineers, and tested as a whole.  But you don't get to mess with the system.

    On the other hand, open source gives you the ability to join the software equivalent of the tuner/modified market - you can tweak the system to your hearts content.  You may make it go faster, but you're not totally sure what it's going to do to the overall quality of the system.

    In fact, I constantly read that that's one of the huge benefits of open source - on an open source project, if you don't like how something works, you can just step in and fix it, while with COTS you don't have that ability.

    Software engineering is software engineering, whether it's open source or closed source.  Having the ability to tweak code (or an automobile) doesn't automatically mean that the tweak will be higher quality than what it's replacing.  It's entirely possible that it either won't be better, or that the improvement won't really matter.  On the IMAP mailing list, I CONSTANTLY see people submitting patches to the U.W. IMAP server proposing tweaks to fix one thing or another (even though it’s the wrong mailing list, the patches still come in).  And Mark Crispin shoots them down all the time, because the person making the patch didn’t really understand the system – their patch might have fixed their problem and their configuration, but it either opened up a security hole, or broke some other configuration, etc.

    Btw, the same thing holds true for system modifications.  Just because you can put a window into a hard disk doesn’t mean that the hard disk is going to work as well afterwards as it did before you took it apart.

    Just like in the automotive world, simply because you CAN modify something, it doesn't mean that it's a good idea to modify it.

  • Larry Osterman's WebLog

    Did you know that OS/2 wasn't Microsoft's first non Unix multi-tasking operating system?


     Most people know about Microsoft’s official timeline for its operating-system like products

    1.      Xenix - Microsoft’s first operating system, which was a version of UNIX that we did for microprocessors. 

    2.      MS-DOS/PC-DOS, a 16 bit operating system for the 8086 CPU

    3.      Windows (not really an operating system, but it belongs in the timeline).

    4.      OS/2, a 16 bit operating system written in joint development with IBM.

    5.      Windows NT, a 32 bit operating system for the Intel i386 processor, the Mips R8800 and the DEC Alpha

    But most people don’t know about Microsoft’s other multitasking operating system, MS-DOS 4.0 (not to be confused with PC-DOS 4.0)

    MS-DOS 4.0 was actually a version of MS-DOS 2.0 that was written in parallel with MS-DOS 3.x (DOS 3.x shipped while DOS 4 was under development, which is why it skipped a version).

    DOS 4 was a preemptive real-mode multitasking operating system for the 8086 family of processors.  It had a boatload of cool features, including movable and discardable code segments, movable data segments (the Windows memory manager was a version of the DOS 4 memory manager).  It had the ability to switch screens dynamically – it would capture the foreground screen contents, save it away and switch to a new window.

    Bottom line: DOS 4 was an amazing product.  In fact, for many years (up until Windows NT was stable), one of the DOS 4 developers continued to use DOS 4 on his desktop machine as his only operating system.

    We really wanted to turn DOS 4 into a commercial version of DOS, but...   Microsoft at the time was a 100% OEM shop – we didn’t sell operating systems, we sold operating systems to hardware vendors who sold operating systems with their hardware.  And in general the way the market worked in 1985 was that no computer manufacturer was interested in a version of DOS if IBM wasn’t interested.  And IBM wasn’t interested in DOS.  They liked the idea of multitasking however, and they were very interested in working with that – in fact, one of their major new products was a product called “TopView”, which was a character mode window manager much like Windows.  The wanted an operating system that had most of the capabilities of DOS 4, but that ran in protected mode on the 286 processor.  So IBM and Microsoft formed the Joint Development Program that shared development resources between the two companies.  And the DOS 4 team went on to be the core of Microsoft’s OS/2 team.

    But what about DOS 4?  It turns out that there WERE a couple of OEMs that had bought DOS 4, and Microsoft was contractually required to provide the operating system to them.  So a skeleton crew was left behind to work on DOS and to finish it to the point where the existing DOS OEM’s were satisfied with it.


    Edit: To fix the title which somehow got messed up.


  • Larry Osterman's WebLog

    Structured Exception Handling Considered Harmful


    I could have sworn that I wrote this up before, but apparently I’ve never posted it, even though it’s been one of my favorite rants for years.

    In my “What’s wrong with this code, Part 6” post, several of the commenters indicated that I should be using structured exception handling to prevent the function from crashing.  I couldn’t disagree more.  In my opinion, SEH, if used for this purpose takes simple, reproducible and easy to diagnose failures and turns them into hard-to-debug subtle corruptions.

    By the way, I’m far from being alone on this.  Joel Spolsky has a rather famous piece “Joel on Exceptions” where he describes his take on exception (C++ exceptions).  Raymond has also written about exception handling (on CLR exceptions).

    Structured exception handling is in many ways far worse than C++ exceptions.  There are multiple ways that structured exception handling can truly mess up an application.  I’ve already mentioned the guard page exception issue.  But the problem goes further than that.  Consider what happens if you’re using SEH to ensure that your application doesn’t crash.  What happens when you have a double free?  If you don’t wrap the function in SEH, then it’s highly likely that your application will crash in the heap manager.  If, on the other hand, you’ve wrapped your functions with try/except, then the crash will be handled.  But the problem is that the exception caused the heap code to blow past the release of the heap critical section – the thread that raised the exception still holds the heap critical section. The next attempt to allocate memory on another thread will deadlock your application, and you have no way of knowing what caused it.

    The example above is NOT hypothetical.  I once spent several days trying to track down a hang in Exchange that was caused by exactly this problem – Because a component in the store didn’t want to crash the store, they installed a high level exception handler.  That handler caught the exception in the heap code, and swallowed it.  And the next time we came in to do an allocation, we hung.  In this case, the offending thread had exited, so the heap critical section was marked as being owned by a thread that no longer existed.

    Structured exception handling also has performance implications.  Structured exceptions are considered “asynchronous” by the compiler – any instruction might cause an exception.  As a result of this, the compiler can’t perform flow analysis in code protected by SEH.  So the compiler disables many of its optimizations in routines protected by try/catch (or try/finally).  This does not happen with C++ exceptions, by the way, since C++ exceptions are “synchronous” – the compiler knows if a method can throw (or rather, the compiler can know if an exception will not throw).

    One other issue with SEH was discussed by Dave LeBlanc in Writing Secure Code, and reposted in this article on the web.  SEH can be used as a vector for security bugs – don’t assume that because you wrapped your function in SEH that your code will not suffer from security holes.  Googling for “structured exception handling security hole” leads to some interesting hits.

    The bottom line is that once you’ve caught an exception, you can make NO assumptions about the state of your process.  Your exception handler really should just pop up a fatal error and terminate the process, because you have no idea what’s been corrupted during the execution of the code.

    At this point, people start screaming: “But wait!  My application runs 3rd party code whose quality I don’t control.  How can I ensure 5 9’s reliability if the 3rd party code can crash?”  Well, the simple answer is to run that untrusted code out-of-proc.  That way, if the 3rd party code does crash, it doesn’t kill YOUR process.  If the 3rd party code is processing a request crashes, then the individual request fails, but at least your service didn’t go down in the process.  Remember – if you catch the exception, you can’t guarantee ANYTHING about the state of your application – it might take days for your application to crash, thus giving you a false sense of robustness, but…


    PS: To make things clear: I’m not completely opposed to structured exception handling.  Structured exception handling has its uses, and it CAN be used effectively.  For example, all NT system calls (as opposed to Win32 APIs) capture their arguments in a try/except handler.  This is to guarantee that the version of the arguments to the system call that is referenced in the kernel is always valid – there’s no way for an application to free the memory on another thread, for example.

    RPC also uses exceptions to differentiate between RPC initiated errors and function return calls – the exception is essentially used as a back-channel to provide additional error information that could not be provided by the remoted function.

    Historically (I don’t know if they do this currently) the NT file-systems have also used structured exception handling extensively.  Every function in the file-systems is protected by a try/finally wrapper, and errors are propagated by throwing exception this way if any code DOES throw an exception, every routine in the call stack has an opportunity to clean up its critical sections and release allocated resources.  And IMHO, this is the ONLY way to use SEH effectively – if you want to catch exceptions, you need to ensure that every function in your call stack also uses try/finally to guarantee that cleanup occurs.

    Also, to make it COMPLETELY clear.  This post is a criticism of using C/C++ structured exception handling as a way of adding robustness to applications.  It is NOT intended as a criticism of exception handling in general.  In particular, the exception handling primitives in the CLR are quite nice, and mitigate most (if not all) of the architectural criticisms that I’ve mentioned above – exceptions in the CLR are synchronous (so code wrapped in try/catch/finally can be optimized), the CLR synchronization primitives build exception unwinding into the semantics of the exception handler (so critical sections can’t dangle, and memory can’t be leaked), etc.  I do have the same issues with using exceptions as a mechanism for error propagation as Raymond and Joel do, but that’s unrelated to the affirmative harm that SEH can cause if misused.

  • Larry Osterman's WebLog

    How do I divide fractions?


    Valorie works as a teacher's aid in a 6th grade classroom at a local elementary school.

    They've been working on dividing fractions recently, and she spent about two hours yesterday working with one student trying to explain exactly how division of fractions works.

    So I figured I'd toss it out to the blogsphere to see what people's answers are.  How do you explain to a 6th grader that 1/2 divided by 1/4 is 2? 

    Please note that it's not sufficient to say: Division is the same as multiplication by the inverse, so when you divide two fractions, you take the second one, invert it, and multiply.  That's stating division of fractions as an axiom, and not a reason.

    In this case in particular, the teacher wants the students to be able to graphically show how it works.

    I can do this with addition and subtraction of numbers (both positive and negative) using positions on a number line. Similarly, I can do multiplication of fractions graphically - you have a whole, divide it into 2 halves.  When you multiply the half by a quarter, you are quartering the half, so you take the half, divide it into fours, and one of those fours is the answer.

    But how do you do this for division?

    My wife had to type this part because we have a bit of, um, discussion, about how simple this part is....

    How can you explain to 9-11 year old kids why you multiply by the reciprocal without resorting to the axiom? It's easy to show graphically that 1/2 divided by 1/4 is 2 quarters because the kids can see that there are two quarters in one half. Equally so, the kids can understand that 1/4 divided by 1/2 is 1/2 of a half because the kids can see that only half of the half is covered by the original quarter. The problem comes in when their intuition goes out.  They can solve it mathematically, but the teacher is unwilling to have them do the harder problems “on faith“ and the drawing is really confusing the kids. Having tried to draw the 5/8 divided by 3/10, I can assure you, it is quite challenging. And no, the teacher is not willing to keep the problems easy. And no, don't get me started on that aspect of this issue.

    I'm a big fan that if one method of instruction isn't working, I try to find another way to explain the concept. I visited my usual math sites and found that most people don't try to graph this stuff until 10th grade or adulthood. Most of the sites have just had this “go on faith“ response (show the kids the easy ones, and let them “go on faith“ that it will hold true for all cases). I really wish I could figure out a way to show successive subtraction, but even that gets difficult on the more complicated examples.

    What I am hoping is that someone out there can provide me with the “aha!“ I need to come up with a few more ways to explain this. What this has been teaching me is that I've been doing this “on faith“ most of my life and never stopped to think about why myself.

    Any ideas/suggestions would be much appreciated.


  • Larry Osterman's WebLog

    How do I open ports in the Windows Firewall?


    One of the side-projects I recently was assigned to work on was to switch the Windows Media Connect project from using the home-brewed HTTP server that was originally coded for the product, to using HTTP.SYS, which is included in XP SP2.  This was as a part of a company-wide initiative to remove all home-brewed HTTP servers (and there were several) and replace them with a single server.  The thinking was that having a half dozen HTTP servers in the system was a bad idea, because each of them was a potential attack vector.  Now with a single server, we have the ability to roll out fixes in a single common location.

    The HTTP.SYS work was fascinating, and I’ll probably write about it more over time, but I wanted to concentrate on a single aspect of the problem.

    I got the server working relatively quickly until we picked up a new version of XP SP2.  That one featured additional improvements to the firewall, and all of a sudden, the remote devices couldn’t retrieve content from the web server.  The requests weren’t getting to our service at all.  What was weird was that they WERE getting the content directory (the names of the files on the machine) but when they tried to retrieve them, they failed. 

    Well, we had suspected that this was going to happen; the new build of SP2 moved HTTP.SYS behind the firewall (it had been in front of the firewall previously).  So now we needed to open a hole in the firewall for our process, the UPnP hosting service had already opened their port, that's why the content directory was available.  Over the next several posts, I’ll go through the process that I went through do discover how to do this.  Everything I needed was documented, but it wasn’t always obvious. 

    The first thing we had to deal with was the fact that we only wanted to open the firewall on local subnet addresses.  To prevent users’ multimedia content from going outside their home, WMC will only accept connections from IP addresses that are in the private network IP address range (192.168.x.x) and the AutoIP address range (169.254.x.x).  We also open up the local address ranges of 10.x.x.x and 172.16.x.x (with a netmask of 0xff, 0xf0, 0, 0) .  So we only wanted to open the firewall on private IP addresses,  it would be a “bad” thing if we opened the WMC port to public addresses, since that could potentially be used as an attack vector.

    The Windows firewall has been documented since Windows XP, the first MSDN hit for “internet connection firewall” returns this page that documents the API.  For XP SP2, there’s a new firewall API, if you use an MSDN search for “firewall API” the first hit is this page which describes the XP SP2 firewall API in great detail.  For a number of reasons (chief among which was that when I wrote the code the firewall API hadn’t been published), my implementation uses the original firewall API that’s existed since Windows XP.  As a result, my code and the techniques I’ve described in the next couple of posts work should work just fine on Windows XP as well as working on XP SP2.

    Anyway, on with the story.  So, as always, I started with the API documentation.  After groveling through the API for a while, I realized I was going to need to use the INetSharingConfiguration interface’s AddPortMapping API to add the port.  I’d want to use the INetSharingConfiguration API on each of the IP addresses that WMC was using.

    So to add a mapping for my port, I simply called INetSharingConfiguration::AddPortMapping specifying a name (in my case I used the URL for the IP address), internal and external port (the same in my case), and a string with the local IP address.  That API returned an INetSharingPortMapping object, which we have to Enable to make it effective.

    Tomorrow: How do we get the INetSharingConfiguration?

    Edit: Clarified IP addresses used for WMC after further investigation.

    Edit: Updated link


  • Larry Osterman's WebLog

    APIs you never heard of - the Timer APIs


    It's time for another "APIs you never heard of" article :)

    This time, I'd like to talk about the time* APIs.

    The time* APIs are a set of 7 APIs built into the windows multimedia extensions (winmm.dll).  They provide a rudimentary set of timer functions for Windows applications.  At this point, except for two of the APIs, they exist only for historical purposes, the core OS now provides significantly higher quality APIs for timers.

    The time APIs fall into three rough categories:

    1. Timer information (timeGetDevCaps, timeGetTime and timeGetSystemTime)
    2. Timer callback functions (timeSetEvent and timeKillEvent)
    3. Timer frequency functions (timeBeginPeriod and timeEndPeriod)

    The first two categories are obsolete (arguably timeGetDevCaps still has valid uses).  The timeGetTime API is effectively identical to the GetTickCount() API, and timeGetSystemTime simply returns the exact same value that timeGetTime would have returned, packed into a MMTIME structure. 

    The timeSetEvent and timeKillEvent have been replaced with the Win32 Timer Queue functions, I'm not sure if I know of any reason to ever call the MME versions of these functions :).  In fact, timeSetEvent will call PulseEvent API, which is fundamentally flawed.  There is one difference between timeSetEvent and the Win32 timer queue functions - timeSetEvent will call timeBeginPeriod to set the timer resolution to the resolution specified in the call to timeSetEvent.  Even with this, you're better off calling timeBeginPeriod and using the Win32 Timer Queue functions (because the Win32 timer queue functions are far more flexible). 

    But then there's the timeBeginPeriod and timeEndPeriod APIs.  These are actually fun APIs, especially in the multimedia or gaming space, because they allow you to change the resolution of the internal scheduler, which can lower (or raise) the resolution with which the internal clock runs.

    This has a number of side effects - it increases the responsiveness of the system to periodic events (when event timeouts occur at a higher resolution, they expire closer to their intended time).  But that increased responsiveness comes at a cost - since the system scheduler is running more often, the system spends more time scheduling tasks, context switching, etc.  This can ultimately reduce overall system performance, since every clock cycle the system is processing "system stuff" is a clock cycle that isn't being spent running your application.  For some multimedia applications (video, for example) the increased system responsiveness is worth the system overhead (for instance, if you're interested in very low latency audio or video, you need the system timers to run at a high frequency).

    Edit: Added comment about timeSetEvent calling timeBeginPeriod to set the resolution.\

    Edit2 (years later): Updated the link to GetTickCount...

  • Larry Osterman's WebLog

    Breaking Up (shared services) Is(n't) Hard To Do

    The last time I wrote, I talked about shared services. One of the problems of working with shared services is that sometimes one service in the process gets in the way of other services.

    For the audio service, it lives in the "networking services" service host (because the networking services svchost is used for all services that run as LocalSystem).  But, because it runs in the same process as the networking functionality, it means that it can be quite difficult to debug the audio service, especially if you're using a source level debugger - if your debugger has to talk to the network, and portions of the networking stack are suspended (by the debugger) it can be hard to make things work...

    It turns out that there's a remarkably clever trick that can be used to split a normally shared service into its own process. 

    From a Windows command prompt, simply type:

    C:\>sc config <servicename> type= own

    To move the service back into its normal shared config, type:

    C:\>sc config <servicename> type= share

    The SC tool should be in the system32 directory on all XP installations, if not, it's in the platform SDK (I believe), and obviously you need to be an administrator to make this work.

    I can't take credit for this, it was shown to me by one of the NT perf guys, but I like it sufficiently that it's worth sharing.


    One more caveat: Before people start trying this on their XP system, please note that there's a reason that those services are in the same process.  Splitting them up will cause your system to use a LOT more memory, and WILL make your system unstable.  I'm posting this trick because it can be quite useful for people who are developing their own shared services.

    Several of the built-in services assume that they're in the same address space as other services, and if they start running in separate processes, they will crash in strange and mysterious ways, if you're not careful, you can render your machine unbootable.

    Just don't go there, it wouldn't be prudent.

  • Larry Osterman's WebLog

    So you need a worker thread pool...


    And, for whatever reason, the NT’s built-in thread pool API doesn’t work for you.

    Most people would write something like the following (error checking removed to reduce typing (and increase clarity)):

    class WorkItem
        LIST_ENTRY m_listEntry;

    class WorkerThreadPool
        HANDLE m_heventThreadPool;
        CRITICAL_SECTION m_critsThreadPool;
        LIST_ENTRY m_workItemQueue;

        void QueueWorkItem(WorkItem *pWorkItem)
            //   Insert the work item onto the work item queue.
            InsertTailList(&m_workItemQueue, pWorkItem->m_listEntry);
            //   Kick the worker thread pool
        void WorkItemThread()
            while (1)
                // Wait until we’ve got work to do
                WaitForSingleObject(&m_heventThreadPool, INFINITE);
                //  Remove the first item from the queue.
                workItem = RemoveHeadList(&m_workItemQueue);
                // Process the work item if there is one.
                if (workItem != NULL)
                    <Process Work Item>

    I’m sure there are gobs of bugs here, but you get the idea.  Ok, what’s wrong with this code?  Well, it turns out that there’s a MASSIVE scalability problem in this logic.  The problem is the m_critsWorkItemQueue critical section.  It turns out that this code is vulnerable to condition called “lock convoys” (also known as the “boxcar” problem).  Basically the problem occurs when there are more than one threads waiting on the m_heventThreadPool event.  What happens when QueueWorkItem calls SetEvent on the thread pool event?  All the threads in the thread pool immediately wake up and block on the work queue critical section.  One of the threads will “win” and will acquire the critical section, pull the work item off the queue and release the critical section.  All the other threads will then wake up, one will successfully acquire the critical section, and all the others will go back to sleep.  The one that woke up will see there’s no work to do and will block on the thread pool.  This will continue until all the work threads have made it past the critical section.

    Essentially this is the same situation that you get when you have a bunch of boxcars in a trainyard.  The engine at the front of the cars starts to pull.  The first car moves a little bit, then it stops because the slack between its rear hitch and the front hitch of the second car is removed.  And then the second car moves a bit, then IT stops because the slack between its rear hitch and the front hitch of the 3rd card is removed.  And so forth – each boxcar moves a little bit and then stops.  And that’s just what happens to your threads.  You spend all your valuable CPU time executing context switches between the various threads and none of the CPU time is spent actually processing work items.

    Now there are lots of band-aids that can be applied to this mechanism to make it smoother.  For example, the m_heventThreadPool event could be an auto-reset event, which means that only one thread would wake up for each work item.  But that’s only a temporary solution - if you get a flurry of requests queued to the work pool, you can still get multiple worker threads waking up simultaneously.

    But the good news is that there’s an easier way altogether.  You can use NT’s built-in completion port logic to manage your work queues.  It turns out that NT exposes a really nifty API called PostQueuedCompletionStatus that essentially lets NT manage your worker thread queue for you!

    To use NT’s completion ports, you create the port with CreateIoCompletionPort, remove items from the completion port with GetQueuedCompletionStatus and add items (as mentioned above) with PostQueuedCompletionStatus.

    PostQueuedCompletionStatus takes 3 user specified variables, one of which which can be used to hold a 32 bit integer (dwNumberOfBytesTransferred), and two of which can be used to hold pointers (dwCompletionKey and lpOverlapped).  The contents of these parameters can be ANY value; the API blindly passes them through to GetQueuedCompletionStatus.

    So, using NT’s completion ports, the worker thread class above becomes:

    class WorkItem

    class WorkerThreadPool
        HANDLE m_hcompletionPort;

        void QueueWorkItem(WorkItem *pWorkItem)
            PostQueuedCompletionStatus(m_hcompletionPort, 0, (DWORD_PTR)pWorkItem, NULL);

        void WorkItemThread()
            while (1)
                GetQueuedCompletionStatus(m_hCompletionPort, &numberOfBytes, &pWorkItem, &lpOverlapped, INFINITE);
                // Process the work item if there is one.
                if (pWorkItem != NULL)
                    <Process Work Item>

    Much simpler.  And as an added bonus, since NT’s managing the actual work queue in the kernel, it allows NT to eliminate the lock convoy in the first example.


    [Insert std disclaimer: This posting is provided "AS IS" with no warranties, and confers no rights]

  • Larry Osterman's WebLog

    My office guest chair


    Adam writes about his office guest chair.

    Microsoft's a big company, and, like all big company has all sorts of silly rules of what you can have in your office.   One of them is that for office furniture, you get:

    1. A desk chair
    2. One PED (sort of a mobile filing cabinet)
    3. One curved desk piece (we have modular desk pieces with adjustable heights)
    4. One short straight desk piece
    5. One long straight desk piece
    6. One white board
    7. One cork board
    8. One or two hanging book shelves with THREE shelves (not 4)
    9. One guest chair.

    If you're a manager, you can get a round table as well (presumably to have discussions at).

    In my case, most of my office stuff is pretty stock - except I got my manager to requisition a round table for his office for me (he already had one).  I use it to hold my manipulative puzzles.  I also have two PEDs

    But I'm most proud of my guest chair.  I have two of them.  One's the standard Microsoft guest chair.  But the other one's special.  You see, it comes from the original Microsoft campus at 10700 Northup Way, and is at least 20 years old.

    I don't think that it's the original chair I had in my original office way back then - that was lost during one of my moves, but I found the exact match for the chair in a conference room the day after the move and "liberated" it. 

    But I've had this particular chair since at least 1988 or so.  The movers have dutifly moved it with me every time.

    Daniel loves it when he comes to my office since it's comfy - it's padded and the standard guest chairs aren't.

    Edit: Someone asked me to include a picture of the chair:

  • Larry Osterman's WebLog

    The application called an interface that was marshalled for a different thread.

    Another one from someone sending a comment:

    I came across your blog and was wondering if what to do when encountering above error message in an application program.

    The error occurs once in a while when printing out from a Windows application.

    Is there some setting missing in computer administration or in the Registry or can it only be solved in the code?

    Appreciate your help!

    Yech.  This one's ugly.  It's time for Raymond's Psychic Powers(tm) of detection.

    If you take the error message text and look inside winerror.h, you'll see that the error message mentioned is exactly the text for the RPC_E_WRONG_THREAD error.

    If you then do an MSDN search for RPC_E_WRONG_THREAD, the first hit is: "INFO: Explanation of RPC_E_WRONG_THREAD Error".  Essentially, the error's a side effect of messing up threading models.  I wrote about them about 18 months ago in "What are these threading models, and why do I care?". 

    So, knowing that the app's dutifully reporting RPC_E_WRONG_THREAD to the user, what had to have happened to cause this error?

    It means that the application did a CoCreateInstance of an Single Threaded Apartment COM object in one thread, but used it in another thread.

    Given the comment that it only happens once in a while, we can further deduce that the application called CoCreateInstance from a thread in a pool of worker threads, and attempted to use it in a function queued to that pool of threads (otherwise it would fail all the time and the author of the app would have found the problem).  Given that it only happens when printing (an operation that's usually handled in a background thread), this makes sense.

    Unfortunately for the person who asked the question, they don't really have any choice but to contact the vendor that created the app and hope that they have an update that fixes the problem, because there's no workaround you can do outside the app :(

  • Larry Osterman's WebLog

    Still more misinformation about virtual memory and paging files


    The wired network in my building's being unusually flakey so I'm posting this from my laptop, sorry for the brevety..

    Slashdot had a front page story today about an article be Adrian Wong posted in his Rojak Pot: "Virtual Memory Optimization Guide".

    I've not finished reading it (the site's heavily slashdotted), but his first paragraph got me worried:

    Back in the 'good old days' of command prompts and 1.2MB floppy disks, programs needed very little RAM to run because the main (and almost universal) operating system was Microsoft DOS and its memory footprint was small. That was truly fortunate because RAM at that time was horrendously expensive. Although it may seem ludicrous, 4MB of RAM was considered then to be an incredible amount of memory.

    4MB of RAM?  Back in the "good old days" of 1.2MB floppy disks (those were the 5 1/4" floppy drives in the PC/AT) the most RAM that could be addressed by a DOS based computer was 1M.  If you got to run Xenix-286, you got a whopping 16M of physical address space.

    I was fuming by the time I'd gotten to the first sentence paragraph of the first section:

    Whenever the operating system has enough memory, it doesn't usually use virtual memory. But if it runs out of memory, the operating system will page out the least recently used data in the memory to the swapfile in the hard disk. This frees up some memory for your applications. The operating system will continuously do this as more and more data is loaded into the RAM.

    This is SO wrong on so many levels.  It might have been be true for an old (OS8ish) Mac, but it's not been true for any version of Windows since Windows 95.  And even for Windows 1.0, the memory manager didn't operate in that manner (it it was a memory manager but it didn't use virtual memory (it was always enabled and active swapping data in and out of memory, but the memory manager didn't use the hardware (since there wasn't any hardware memory management for Windows 1.0))).

    It REALLY disturbs me when articles like this get distributed.  Because it shows that the author fundimentally didn't understand what he's writing about (sort-of like what happens when I write about open source :) - at least nobody's ever quoted me as an authority on that particular subject)

    Edit: I'm finally at home, and I've had a chance to read the full article.  I've not changed my overall opinion of the article, as a primer on memory management, it's utterly pathetic (and dangerously incorrect).  Having said that, the recommendations for improving the performance of your paging file are roughly the same as I'd come up with if I was writing the article.  Most importantly, he differentiates between the difference between having a paging file on a partition and on a separate drive, and he adds some important information on P-ATA and RAID drive performance characteristics that I wouldn't have included if I was writing the article.  So if you can make it past the first 10 or so pages, the article's not that bad.


  • Larry Osterman's WebLog

    How to lose customers without really trying...


    Not surprisingly, Valorie and I both do some of our holiday season shopping at ThinkGeek.  But no longer.  Valorie recently placed a substantial order with them, but Instead of processing her order, they sent the following email:

    From: ThinkGeek Customer Service []
    Sent: Thursday, November 15, 2007 4:28 AM
    To: <Valorie's Email Address>
    Subject: URGENT - Information Needed to Complete Your ThinkGeek Order

    Hi Valorie,

    Thank you for your recent order with ThinkGeek, <order number>. We would like to process your order as soon as possible, but we need some additional information in order to complete your order.

    To complete your order, we must do a manual billing address verification check.

    If you paid for your order via Paypal, please send us a phone bill or other utility bill showing the same billing address that was entered on your order.

    If you paid for your order via credit card, please send us one of the following:

    - A phone bill or other utility bill showing the same billing address that was entered on your order

    - A credit card statement with your billing address and last four digits of your credit card displayed

    - A copy of your credit card with last four digits displayed AND a copy of a government-issued photo ID, such as a driver's license or passport.

    To send these via e-mail (a scan or legible digital photo) please reply to or via fax (703-839-8611) at your earliest convenience. If you send your documentation as digital images via email, please make sure they total less than 500kb in size or we may not receive your email. We ask that you send this verification within the next two weeks, or your order may be canceled. Also, we are unable to accept billing address verification from customers over the phone. We must receive the requested documentation before your order can be processed and shipped out.

    For the security-minded among you, we are able to accept PGP-encrypted emails. It is not mandatory to encrypt your response, so if you have no idea what we're talking about, don't sweat it. Further information, including our public key and fingerprint, can be found at the following


    At ThinkGeek we take your security and privacy very seriously. We hope you understand that when we have to take extra security measures such as this, we do it to protect you as well as ThinkGeek.

    We apologize for any inconvenience this may cause, and we appreciate your understanding. If you have any questions, please feel free to email or call us at the number below.


    ThinkGeek Customer Service

    1-888-433-5788 (phone)

    1-703-839-8611 (fax)

    Wow.  We've ordered from them in the past (and placed other large orders with them), but we've never seen anything as outrageous as this.  They're asking for exactly the kind information that would be necessary to perpetuate an identity theft of Valorie's identity, and they're holding our order hostage if we don't comply.

    What was worse is that their order form didn't even ask for the CVE code on the back of the credit card (the one that's not imprinted).  So not only didn't they follow the "standard" practices that most e-commerce sites follow when dealing with credit cards, but they felt it was necessary for us to provide exactly the kind of information that an identity thief would ask for.

    Valorie contacted them to let them know how she felt about it, and their response was:

    Thank you for your recent ThinkGeek order. Sometimes, when an order is placed with a discrepancy between the billing and the shipping addresses, or with a billing address outside the US, or the order is above a certain value, our ordering system will flag the transaction. In these circumstances, we request physical documentation of the billing address on the order in question, to make sure that the order has been placed by the account holder. At ThinkGeek we take your security and privacy very seriously. We hope you understand that when we have to take extra security measures such as this, we do it to protect you as well as ThinkGeek.
    Unfortunately, without this documentation, we are unable to complete the processing of your order. If we do not receive the requested documentation within two weeks of your initial order date, your order will automatically be cancelled. If you can't provide documentation of the billing address on your order, you will need to cancel your current order and reorder using the proper billing address for your credit card. Once we receive and process your documentation, you should not need to provide it on subsequent orders. Please let us know if you have any further questions.

    The good news is that we have absolutely no problems with them canceling the order, and we're never going to do business with them again.  There are plenty of other retailers out there that sell the same stuff that ThinkGeek does who are willing to accept our business without being offensive about it.


    Edit to add:  Think Geek responded to our issues, their latest response can be found here.

  • Larry Osterman's WebLog

    Why add a throw() to your methods?


    Towards the end of the comments in my last "What's wrong with this code" , hippietim asked why I added a throw()attribute to the destructor of the CCoInitializer.  The answer's pretty simple.  If you add a throw() attribute around routines that never throw, the compiler can be clever about code motion and optimization.  Consider the following totally trivial:

    class MyClass
        size_t CalculateFoo()
        size_t MethodThatCannotThrow()
    return 100;
    void ExampleMethod()
            size_t foo, bar;
                foo = CalculateFoo();
                bar = foo * 100;
    "bar is %d", bar);
            catch (...)


    When the compiler sees this, with the "throw()" attribute, the compiler can completely optimize the "bar" variable away, because it knows that there is no way for an exception to be thrown from MethodThatCannotThrow().  Without the throw() attribute, the compiler has to create the "bar" variable, because if MethodThatCannotThrow throws an exception, the exception handler may/will depend on the value of the bar variable.

    In addition, source code analysis tools like prefast can (and will) use the throw() annotation to improve their error detection capabilities - for example, if you have a try/catch and all the functions you call are marked as throw(), you don't need the try/catch (yes, this has a problem if you later call a function that could throw).

  • Larry Osterman's WebLog

    What are Known DLLs anyway?


    In my previous post about DLLs and how they work, I commented that winmm.dll was a KnownDLL in Longhorn.  It turns out that this is a bug in an existing KnownDLL. But what in the heck ARE Known DLLs in the first place?

    Well, it turns out that it’s in the KB, and I’ll summarize.

    KnownDLL’s is a mechanism in Windows NT (and win9x) that allows the system to “cache” commonly used system DLLs.  It was originally added to improve application load time, but it also can be considered a security mechanism, since it prevents people from exploiting weak application directory permissions by dropping in Trojan horse versions of system DLLs (since the key system DLLs are all known DLLs, the version of the file in the application directory will be ignored).  As a security mechanism it's not a particularly strong mechanism (if you can write to the directory that contains a program, you can create other forms of havoc), but it can be considered a security mechanism.

    If you remember from my previous article, when the loader finds a DLL import record in an executable, it opens the file and tries to map the file into memory.  Well, that’s not ENTIRELY the case.  In fact, before that happens the loader looks for an existing section called \KnownDlls\<dll filename>.  If that section exists, then instead of opening the file, the loader simply uses the existing section.   It then follows all the “normal” rules for loading a DLL.

    When the system boots, it looks in the registry at HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Session Manager\KnownDLLs and creates a \KnownDlls\<dll filename> section for every DLL listed under that registry key.

    If you compare the HKLM\System\CCS\Control\Session Manager\KnownDLLs registry key with the sections under \KnownDlls (using a viewer like winobj), you’ll notice that the \KnownDlls object container always has more entries in it than the registry key.  This is because the \KnownDlls sections are computed as the transitive closure of the DLLs listed in KnownDLLs.  So if a DLL’s listed in KnownDLLs, all of the DLL’s that are statically linked with the DLL are ALSO listed in the \KnownDlls section.

    Also, if you look in the KnownDLLs registry key, you’ll notice that there’s no path listed for the KnownDLLs.  That’s because all KnownDLLs are assumed to be in the directory pointed to by HKLM\System\CCS\Control\KnownDLLs\DllDirectory registry value.  Again, this is an aspect of KnownDLLs being a security feature – by requiring KnownDLLs to be in the same directory, it makes it harder for someone to inject their own Trojan version of one of the KnownDLLs.

    Oh, and if the KnownDLLs processing causes any issues, or if for some other reason you don't want the system to load a DLL as a KnownDll, then you can set HKLM\System\CCS\Control\Session Manager\ExcludeFromKnownDlls to exclude a DLL from the KnownDll processing.  So in my example, until the bug is fixed in the existing KnownDLL, I’m adding winmm.dll to my ExcludeFromKnownDlls list.



  • Larry Osterman's WebLog

    Mirra, first impressions


    We've currently got something like 7 computers currently in use in my house these days, and I've been looking for a centralized backup solution for the home for a while.

    Eventually, I settled on a Mirra - a small form-factor appliance-like backup server.  It comes in four sizes, 80GB, 160GB, 250GB, and 400GB.  I ordered the 400GB based on the amount of stuff we've got saved on the various machines that will have to be backed up.

    I've not yet had a chance to use all the features of the product (in particular, I've not used the remote access functionality), but I  did set it up and get it running on two of the machines over the weekend.

    I have to say that I'm impressed.  IMHO, these guys have been taking lessons from Apple in terms of out-of-box experience (Personally, I think that Apple does OOBE better than any other PC hardware company).

    You open the Mirra box, and you see a cardboard inset, with a folded card-stock flyer and the power cord and an ethernet cord.

    On the cover of the flyer, are the words "Welcome to Mirra".  You open it up, and it unfolds into a four page story telling you that you're about to enter into a new world where you don't have to worry about your data.  On the back of each of the four pages is one of the four steps to setting up the Mirra - the first tab has you plugging in the appliance (you need to plug it into AC and into an ethernet port), the second tab has you installing the software on your PC, the third has you configuring the PC, the fourth is "Relax".

    I LOVED this experience - it's exactly the balance that computer-related appliance should strike - simple instructions, clearly spelled out, easy for Abby to get right.  The actual Mirra device is a small form-factor PC, I didn't crack the case to see what was running inside it, but it's got video, keyboard, mouse, and USB ports on the case (the video and USB are covered over with plastic).  The small form-factor PC is perfect for "plug it in and forget about it".

    I had some difficulties getting the software installed on the first machine I tried, it didn't recognize the firewall I was running (Microsoft One-Care Beta1), and I had to manually configure it.  On the other hand, the manufacturers web site was extremely helpful getting past this hurdle, and once set up, it immediately started copying files.

    I next went to each of the four accounts on that machine and set the software up on each of them.  It worked seamlessly for all four accounts, including all the limited user accounts.  This alone impressed the heck out of me - there aren't that many software products out there that consider the FUS (fast user switching) and LUA scenarios, but clearly the Mirra guys had.

    I then went upstairs to my computer, and installed it.  This machine doesn't have One-Care installed on it, and the Mirra detected the XP SP2 firewall and opened the relevant ports in the firewall (the firewall is enabled and I didn't need to do anything about it).  The machine then started hammering our home network copying off all the files on the machine.

    I still need to get it installed on the kids computers, that'll be interesting since the kids computers don't have access to the internet.

    The Mirra backup software runs as two services, running in two processes (I'm not sure why, since both services run at localsystem).  However, once configured, the Mirra backup software will run without requiring any process in the user's session.  If I was doing it, I'd have used just one process to run both services, but...

    As I commented to Valorie "This is the software I would have designed".  I was utterly impressed that they seem to have nailed several critical scenarios that are usually overlooked.

    One negative (not relevant to me, but probably to others) is that this is a Windows-only product - they don't seem to have Mac or Linux clients.

    In general, though, I'm pretty impressed.


  • Larry Osterman's WebLog

    Why doesn't Mozilla (Firefox) like the Microsoft OCA web site?


    In my previous post about OCA, the comments thread has a long discussion started by Shannon J Hager about Mozilla’s behavior when you attempt to access  If you attempt to access this web site using Firefox (or other Mozilla variants), you get the following dialog box:

    Which is weird, because of course the web site works just fine in IE.  No big deal, right – Microsoft’s well known for sleazing the rules for it’s own products, so obviously this is Microsoft’s fault – they probably did something like hard coding in trust to the Microsoft issuing CA.  But I was kinda surprised at this, so I spent a bit of time checking it out...

    The way that SSL certificate verification is supposed to work is that if the issuer of a certificate isn’t trusted, then the code validating the certificate is supposed to check the parent of the issuer to see if IT is trusted.  If the parent of the issuer isn’t trusted, it’s supposed to check the grandparent of the issuer, and so forth until you find the root certificate authority (CA).

    The issuing CA of the certificate on the winqual web site is the “Microsoft Secure Server Authority”, it’s not surprising Mozilla doesn’t trust that one.  The parent of the issuing CA is the “Microsoft Internet Authority”, again, no surprise that Mozilla doesn’t trust it.

    But the grandparent of the issuing CA is the “GTE CyberTrust Root”.  This is a well known CA, and Mozilla should be trusting it.  And what do you know, Mozilla DOES claim to trust that root CA:

    Well, Cesar Eduardo Barros actually went and checked using openssl to see why the CA isn’t trusted.  He tried:

    $ openssl s_client -connect -showcerts

    depth=0 /C=US/ST=Washington/L=Redmond/O=WHDC (Old WHQL)/OU=Microsoft/
    verify error:num=20:unable to get local issuer certificate
    verify return:1
    depth=0 /C=US/ST=Washington/L=Redmond/O=WHDC (Old WHQL)/OU=Microsoft/
    verify error:num=27:certificate not trusted
    verify return:1
    depth=0 /C=US/ST=Washington/L=Redmond/O=WHDC (Old WHQL)/OU=Microsoft/
    verify error:num=21:unable to verify the first certificate
    verify return:1
    Certificate chain
    0 s:/C=US/ST=Washington/L=Redmond/O=WHDC (Old WHQL)/OU=Microsoft/
    i:/DC=com/DC=microsoft/DC=corp/DC=redmond/CN=Microsoft Secure Server Authority
    -----END CERTIFICATE-----
    Server certificate
    subject=/C=US/ST=Washington/L=Redmond/O=WHDC (Old WHQL)/OU=Microsoft/
    issuer=/DC=com/DC=microsoft/DC=corp/DC=redmond/CN=Microsoft Secure Server Authority
    No client certificate CA names sent
    SSL handshake has read 1444 bytes and written 324 bytes
    New, TLSv1/SSLv3, Cipher is RC4-MD5
    Server public key is 1024 bit
    Protocol : TLSv1
    Cipher : RC4-MD5
    Session-ID: [...]
    Master-Key: [...]
    Key-Arg : None
    Start Time: [...]
    Timeout : 300 (sec)
    Verify return code: 21 (unable to verify the first certificate)

    Decoding the certificate it gave me above (openssl x509 -text) I get the same information Mozilla gives me and a bit more, but no copy of the issuer. The only suspicious thing in there is:

    Authority Information Access:
    CA Issuers - URI:
    CA Issuers - URI:http://corppki/aia/msssa1(1).crt

    Getting that URI gives me a blank HTML page with a 0.1 second redirect to itself. (The CRL one seems valid, however.)

    So I was confused, why wasn’t openSSL able to verify the certificate?  So I started asking the security PM’s here at Microsoft what was up.  One of the things he told me was that Microsoft doesn’t hard code ANY intermediate certificates in our browser.  Instead, our browser relies on the referral information in the certificate to chase down the CA hierarchy.

    So why can’t Mozilla do the same thing?  Is there something wrong with our certificates that’s preventing this from working?  I kept on pestering and the PM’s kept on digging.  Eventually I got email from someone indicating “IE is chasing 48.2 AIA”.

    Well, this isn’t very helpful to me, so I asked the security PM in question to explain it in English.  Apparently the root cause of the problem is that IE is following the Authority Information Access 48.2 OID ( to find the parent of the certificate, while Mozilla isn’t.

    Inside the Microsoft certificate is the following:

    And if you go to you’ll find the parent CA for the certificate on the winqual web site.  So now it’s off to figure out if the IE behavior is according to standard, or if it’s another case of Microsoft ignoring web standards in favor of proprietary extensions.

    A few minutes of googling discovers that the AIA 48.2 field is also known as the id-ad-caIssuers OID.  The authoritative reference for this OID is RFC2459 (the RFC that defines the x.509 certificate infrastructure).  It describes this field as:

     The id-ad-caIssuers OID is used when the additional information lists CAs that have issued certificates superior to the CA that
    issued the certificate containing this extension. The referenced CA Issuers description is intended to aid certificate users in
    the selection of a certification path that terminates at a point trusted by the certificate user.

    In other words, IE is correctly chasing the AIA 48.2 references in the certificate to find the root issuing CA of the certificate. Since it didn’t have direct knowledge of the issuing CA, it correctly looked at the AIA 48.2 field of the certificate for the winqual web site and chased the AIA 48.2 references to the root CA.  It appears that Mozilla (and OpenSSL and GnuSSL) apparently don’t follow this link, which is why they pop up the untrusted certificate dialog.

    Issue solved.  Now all someone has to do is to file bugs against Mozilla and OpenSSL to get them to fix their certificate validation logicJ.

    Btw, I want to give HUGE kudo’s to Cesar Eduardo Barros for tirelessly trying to figure this out, and to Michael Howard and the lead program manager for NT security for helping me figure this out.  If you look at the info from the certificate that Cesar posted above, he correctly caught the AIA 48.2 fields inside the CA, it was a huge step in the right direction, all that remained was to figure out what it really meant.

    Edit: Fixed picture links.

    Edit2: Fixed line wrapping of reference from RFC2459.

  • Larry Osterman's WebLog

    Everyone wants a shiny new UI


    Surfing around the web, I often run into web sites that contain critiques of various aspects of Windows UI.

    One of the most common criticisms on those sites is "old style" dialogs.  In other words, dialogs that don't have the most up-to-date theming.  Here's an example I ran into earlier today:


    Windows has a fair number of dialogs like this - they're often fairly old dialogs that were written before new theming elements were added (or contain animations that predate newer theming options).  They all work correctly but they're just ... old.

    Usually the web site wants the Windows team update the dialog to match the newest styling's because the dialog is "wrong".

    Whenever someone asks (or more often insists) that the Windows team update their particular old dialog, I sometimes want to turn around and ask them a question:

    "You get to choose: You can get this dialog fixed OR you can cut a feature from Windows, you can't get both.  Which feature in Windows would you cut to change this dialog?"

    Perhaps an automotive analogy would help explain my rather intemperate reaction:

    One of the roads near my house is a cement road and the road is starting to develop a fair number of cracks in it.  The folks living near the road got upset at the condition of the road and started a petition drive to get the county to repair the road.  Their petition worked and county came out a couple of weeks later and inspected the road and rendered their verdict on the repair (paraphrasing):  We've looked at the road surface and it is 60% degraded.  The threshold for immediate repairs on county roads is 80% degradation.  Your road was built 30 years ago and cement roads in this area have a 40 year expected lifespan.  Since the road doesn't meet our threshold for immediate repair and it hasn't met the end of its lifespan, we can't justify moving this section of road up ahead of the hundreds of other sections of road that need immediate repair.

    In other words, the county had a limited budget for road repairs and there were a lot of other sections of road in the county that were in a lot worse shape than the one near my house.

    The same thing happens in Windows - there are thousands of features in Windows and a limited number of developers who can change those features.   Changing a dialog does not happen for free.  It takes time for the developers to fix UI bugs.  As an example, I just checked in a fix for a particularly tricky UI bug.  I started working on that fix in early October and it's now January.

    Remember, this dialog works just fine, it's just a visual inconsistency.  But it's going to take a developer some amount of time to fix the dialog.  Maybe it's only one day.  Maybe it's a week.  Maybe the fix requires coordination between multiple people (for example, changing an icon usually requires the time of both a developer AND a graphic designer).  That time could be spent working on fixing other bugs.  Every feature team goes through a triage process on incoming bugs to decide which bugs they should fix.  They make choices based on their limited budget (there are n developers on the team, there are m bugs to fix, each bug takes t time to fix on average, that means we need to fix (m*t)/n bugs before we can ship).

    Fixing theming bug like this takes time that could be spent fixing other bugs.  And (as I've said before) the dialog does work correctly, it's just outdated.

    So again I come back to the question: "Is fixing a working but ugly dialog really more important than all the other bugs?"  It's unfortunate but you have to make a choice.


    PS: Just because we have to make choices like this doesn't mean that you shouldn't send feedback like this.   Just like the neighbors complaining to the county about the road, it helps to let the relevant team know about the issue. Feedback like this is invaluable for the Windows team (that's what the "Send Feedback" link is there for after all).  Even if the team decides not to fix a particular bug in this release it doesn't mean that it won't be fixed in the next release.

  • Larry Osterman's WebLog

    New Audio APIs for Vista


    In an earlier post, I mentioned that we totally re-wrote the audio stack for Windows Vista.  Today I want to talk a bit about the APIs that came along with the new stack.

    There are three major API components to the Vista audio architecture:

    • Multimedia Device API (MMDEVAPI) - an API for enumerating and managing audio endpoints.
    • Device Topology - an API for discovering the internals of your audio card's topology.
    • Windows Audio Session API ((WASAPI) - the low level API for rendering audio.

    All the existing audio APIs have been re-plumbed to use these APIs internally, for Vista, all audio goes through these three APIs.  For the vast majority of the existing audio applications, things should "just work"...

    In general, we don't expect that anyone will move to these new APIs, they're documented for completeness reasons, but the reality is that unless you're dealing with extremely low latency audio (sub 20ms), or writing a control panel applet for a specific audio adapter, you're not likely to ever want to deal with them (the new APIs really are very low level APIs - using the higher level APIs is both easier and less error prone).


    MMDEVAPI is the entrypoint API - it's a COM class that allows applications to enumerate endpoints and "activate" interfaces on them.  Endpoints fall into two general types: Capture and Render (You can consider Capture endpionts as microphones and line in, Render endpoints are things like speakers).  MMDEVAPI also allows the user to manage defaults for each of the types. As I write this, are actually three different sets of defaults supported in Vista: "Console", "Multimedia", and "Communications".  "Console" is used for general purpose audio, "Multimedia" is intended for audio playback applications (media players, etc), and "Communications" is intended for voice communications (applications like Yahoo! Messenger, Microsoft Communicator, etc). 

    Windows XP had two sets of defaults (the "default" default and the "communications" default), we're adding a 3rd default type to enable multimedia playback.  Consider the following scenario.  I have a Media Center computer.  The SPDIF output from the audio adapter's connected to my home AV receiver, I have a USB headset that I want to use for VOIP, and there are stereo speakers connected to the machine that I use for day-to-day operations.  We want to enable applications to make intelligent choices when they choose which audio device to use - the default in this scenario is to use the desktop speakers, but we want to allow Communicator (or Messenger, or whatever) to use the headset, and Media Center to use the external receiver.  We may end up changing these sets before Vista ships, but this give a flavor of what we're thinking about.

    MMDEVAPI supports an "activation" design pattern - essentially, instead of calling a class factory to create a generic object, then binding the object to another object, with activation, you can enumerate objects (endpoints in this case) and "activate" an interface on that object.  It's a really convenient pattern when you have a set of objects that may or may not have the same type.

    Btw, you can access the category defaults using wave or mixer messages, this page from MSDN describes how to access them - the console default is accessed via DRVM_MAPPER_PREFERRED_GET and the communications default is accessed via DRVM_MAPPER_CONSOLEVOICECOM_GET.

    Device Topology:

    Personally, I don't believe that anyone will ever use Device Topology, except for audio hardware vendors who are writing custom control panel extensions.  It exists for control panel type applications that need to be able to determine information about the actual hardware. 

    Device Topology exposes collections of parts and the connections between those parts.  On any part, there are zero or more controls, which roughly correspond to the controls exposed by the audio driver.  One cool thing about device topologies is that topologies can connect to other topologies.  So in the future, it's possible that an application running on an RDP server may be able to enumerate and address the audio devices on the RDP client - instead of treating the client as an endpoint, the server might be able to enumerate the device topology on the RDP client and manipulate controls directly on the client.  Similarly, in the future, the hardware volume control for a SPDIF connector might manipulate the volume on an external AV receiver via an external control connection (1394 or S/LINK).

    One major change between XP and Vista is that Device Topology will never lie about the capabilities of the hardware - before Vista, if a piece of hardware didn't have a particular control the system tried to be helpful and provide controls that it thought ought to be there (for instance if a piece of hardware didn't have a volume control, the system helpfully added one).  For Vista, we're reporting exactly what the audio hardware reports, and nothing more.  This is a part of our philosophy of "don't mess with the user's audio streams if we don't have to" - emulating a hardware control when it's not necessary adds potentially unwanted DSP to the audio stream.

    Again, the vast majority of applications shouldn't need to use these controls, for most applications, the functionality provided by the primary APIs (mixerLine, wave, DSound, etc) are going to be more suitable for their needs.


    WASAPI is the "big kahuna" for the audio engine.  You activate WASAPI on an endpoint, and it provides functionality for rendering/capturing audio streams.  It also provides functions to manage the audio clock and manipulate the volume of the audio stream.

    In general, WASAPI operates in two modes.  In "shared" mode, audio streams are rendered by the application and mixed by the global audio engine before they're rendered out the audio device.  In "exclusive" mode, audio streams are rendered directly to the audio adapter, and no other application's audio will play.  Obviously the vast majority of applications will operate in shared mode, that's the default for the wave APIs and DSound.  One relatively common scenario that WILL use exclusive mode is rendering content that requires a codec that's present in the hardware that Windows doesn't understand.  A simple example of this is compressed AC3 audio rendered over a SPDIF connection - if you attempt to render this content, if Windows doesn't have a decoder for this content, then DSound will automatically initialize WASAPI in exclusive mode and will render the content directly to the hardware.

    If your application is a pro audio application, or is interested in extremely low latency audio then you probably want to consider using WASAPI, otherwise it's better to stick with the existing APIs.

    Tomorrow: Volume control (a subject that's near and dear to my heart) :)

Page 2 of 33 (815 items) 12345»