October, 2005

Larry Osterman's WebLog

Confessions of an Old Fogey
  • Larry Osterman's WebLog

    What's wrong with this code, part 15 - the answers

    • 15 Comments
    Well, that was certainly interesting.  My last "what's wrong with this code" post had to do with a real-world bug in some of the notification infrastructure that's used for audio.  I took some sample code done while developing the prototype, and stripped out anything that would have described the actual infrastructure (NDA issues, sorry :)) to make the example.  As a result, except for typos (like one case where I missed some old code, and an incorrect typo), the code is exactly like I saw it.

    As was mentioned in the comments , the actual bug was here:

            hr = HRESULT_FROM_WIN32(TraceEvent(_TraceHandle, (PEVENT_TRACE_HEADER)&notificationHeader._TraceHeader));
     

    The problem is that the HRESULT_FROM_WIN32 macro is defined as:

    #define HRESULT_FROM_WIN32(x) ((HRESULT)(x) <= 0 ? ((HRESULT)(x)) : \
                        ((HRESULT) (((x) & 0x0000FFFF) | (FACILITY_WIN32 << 16) | 0x80000000)))
     

    Unfortunately, that macro evaluates its argument twice - once when determining if the value is less than 0, the second time when forming the actual error code.

    For some APIs without side effects (like GetLastError()), this isn't a big deal, but for APIs that DO have side effects (basically any API that returns a Win32 error code), it can be catastrophic.  The problem we had was relatively minor (we sent doubled notifications), but this pattern can easily result in memory and handle leaks.  Mike Dimmick pointed out that for Vista, this issue has been addressed, there is now an optional in-line version of HRESULT_FROM_WIN32 that doesn't have this problem, unfortunately, this code didn't use it :(

     

    So Kudos and mea culpas:

    First the mea culpas:  There were two typos in the original example - I left in some old code, and didn't get the type of one of my casts correct.

    Kudos: Marvin was the first person who caught that HRESULT_FROM_WIN32 was a macro.  Close on his heals was K.T., Thales and Harold.

    Now for other things that came up in the comments thread.

    A number of people had concerns about the use of the "notificationBuffer = (PNOTIFICATION_BLOCK)(notificationHeader+1)" construct.  The format of the buffer in question has a common structure (a notification_header) at the head, and  a variable structure (the NOTIFICATION_BLOCK) following.  the "notificationHeader+1" returns an appropriately aligned structure located immediately after the NOTIFICATION_HEADER into the buffer.  Given that the block was allocated using sizeof(NOTIFICATION_HEADER), they correctly noticed that if NOTIFICATION_HEADER was misaligned, this could cause significant issues.  The +1 increments the notificationHeader by the number of bytes in a NOTIFICATION_HEADER, Tim Smith called out the line in the C++ standard where it shows that behavior is required.

    A number of other people were confused about the NotificationBlock->BlockSize paradigm.  The idea is that the caller provides a self-describing buffer (NotificationBlock), and this routine packs it into the resulting block.  The NOTIFICATION_BLOCK provided by the caller had to be at least NOTIFICATION_BLOCK bytes long.   There IS an error here, because the caller doesn't provide the length of the block separately, I can't validate that the NotificationBlock->BlockSize is valid - I'm relying on the C++ compilers type safety to handle this - it may be wrong, but...

    Once again, throwing new raised its ugly head, as did several people who wanted me to use exception handling as an error propagation mechanism, as did probing the NotificationBlock parameter for validity.  I've touched on this before - RAII has its own set of issues here, and I always use the non throwing new.  Oh, and someone was upset at the presence of a "goto" in the code.  Would it be better if I used a "do {} while (FALSE);" and then used "break" instead of the goto?  There's no semantic difference between the two.

    There was also a potential issue of arithmetic overflow or underflow - since this function is internal-only, I'm not particularly worried about arithmetic overflow issues - if this was a public API, I would, but our team controls all the users of this function, so...

  • Larry Osterman's WebLog

    Another Surreal moment

    • 2 Comments

    I was listening to NPR while driving around this afternoon.  They had a report by Libby Lewis on "Scooter" Libby.

    Somewhere over in NPR-land, there is a producer that is just laughing his socks off.  I don't know how on EARTH they managed to arrange a Libby Lewis story about Lewis Libby, but...

     

  • Larry Osterman's WebLog

    What's wrong with this code, part 15

    • 37 Comments
    Work's been pretty hectic lately, that's why so few posts this month, but I ran into a real bug in my code recently that I realized would make a GREAT "What's wrong with this code" post.

     

    HRESULT CNotification::GenerateEvent 
       (     
          PNOTIFICATION_BLOCK NotificationBlock
       ) 
    { 
        HRESULT hr = S_OK; 
        BYTE *buffer = NULL; 
        DWORD bufferSize = sizeof(NOTIFICATION_HEADER) + 
                                NotificationBlock->BlockSize; 
        if (NotificationBlock->BlockSize < sizeof(NOTIFICATION_BLOCK)) 
        { 
            hr = E_INVALIDARG; 
            goto Error; 
        } 
        else 
        { 
            buffer = new BYTE[bufferSize]; 
            if (buffer == NULL) 
            { 
                hr = E_OUTOFMEMORY; 
                goto Error; 
            } 
            PNOTIFICATION_HEADER notificationHeader = (PNOTIFICATION_HEADER)buffer; 
            PNOTIFICATION_BLOCK notificationBuffer; 
    
            ZeroMemory(buffer, bufferSize); 
            notificationBuffer = (PNOTIFICATION_BLOCK)(notificationHeader + 1); 
    
            <Fill in the EVENT_TRACE_HEADER within the NOTIFICATION_HEADER structure>
    
            CopyMemory(notificationBuffer, NotificationBlock, NotificationBlock->BlockSize); 
    
            hr = HRESULT_FROM_WIN32(TraceEvent(_TraceHandle, (PEVENT_TRACE_HEADER)&notificationHeader._TraceHeader));
            if (hr != S_OK) 
            { 
                goto Error; 
            } 
        } 
    Cleanup: 
        delete []buffer; 
        return hr; 
    
    Error: 
        goto Cleanup; 
    }

    Pretty straightforward, but it's got a REALLY nasty bug in it (I was SO embarrassed when I found it).

     

    As always, kudos and mea culpas next post.

    Edit1: Fixed typo (mediaBuffer->buffer).  Also fixed NOTIFICATIONBLOCK that should be PNOTIFICATIONBLOCK

  • Larry Osterman's WebLog

    Does Visual Studio make you stupid?

    • 43 Comments

    I know everyone's talking about this, but it IS a good question...

    Charles Petzold recently gave this speech to the NYC .NET users group.

    I've got to say, having seen Daniel's experiences with Visual Basic, I can certainly see where Charles is coming from.  Due partly to the ease of use of VB, and (honestly) a lack of desire to dig deeper into the subject, Daniel's really quite ignorant of how these "computer" thingies work.  He can use them just fine, but he has no understanding of what's happening.

    More importantly, he doesn't understand how to string functions/procedures together to build a coherent whole - if it can't be implemented with a button or image, it doesn't exist...

     

    Anyway, what do you think?

     

  • Larry Osterman's WebLog

    A really quick followup to the "No Easter Eggs" story

    • 5 Comments

    Man, it's just been the fall for running into old friends and colleagues.  In the past month alone, I've run into three different classmates from college, and co-workers from over the years.  Go figure that one out, I have NO idea why they all caught up in the past month, but it's been REALLY cool..

    Today, I ran into the test lead for the Exchange IMAP and POP3 servers in Seattle Center (Daniel takes a class there every Saturday morning, so I've been there every Saturday for the past four or so years).

    In yesterday's post, I mentioned the Exchange 5.5 IMAP Easter Egg.  Well, I mentioned the story to the test lead, and he (not surprisingly) felt bad that it fired inappropriately.  But he did tell me that Delilah was still around (for those of you who know the Easter Egg, you'll understand).

    Anyway, I thought it was kinda funny :)  Serindipity can be so cool :)

     

  • Larry Osterman's WebLog

    Why no Easter Eggs?

    • 35 Comments

    Yesterday's post caused a bit of a furor in the comments thread.  A large number of people leaving comments (and others) didn't understand why the OS division has a "no Easter Eggs" policy.

    If you think about this, it's not really that surprising.  One of the aspects of Trustworthy Computing is that you can trust what's on your computer.  Part of that means that there's absolutely NOTHING on your computer that isn't planned.  If the manufacturer of the software that's on every desktop in your company can't stop their developers from sneaking undocumented features into the product (even features as relatively benign as an Easter Egg), how can you be sure that they've not snuck some other undocumented feature into the code.

    Even mandating that you have access to the entire source code to the product doesn't guarantee that - first off, nobody in their right mind would audit all 10 million+ lines of code in the product before deployment, and even if you DID have the source code, that doesn't mean anything - Ken Thompson made that quite clear in his Turing Award lecture.  Once you've lost the trust of your customers, they're gone - they're going to find somewhere else to take their business.

    And there are LOTS of businesses and governments that have the sources to Microsoft products.  Imagine how they'd react if (and when) they discovered the code?  Especially when they were told that it was a "Special Surprise" for our users.  Their only reaction would be to wonder what other "Special Surprises" were in the code.

    It's even more than that.  What happens when the Easter Egg has a security bug in it?  It's not that unplausable - the NT 3.1 Easter Egg had a bug in it - the easter egg was designed to be triggered when someone typed in I LOVE NT, but apparently it could also be triggered by any anagram of "I LOVE NT" - as a result, "NOT EVIL" was also a trigger.

    Going still further, Easter Eggs are percieved as a symptom of bloat, and lots of people get upset when they find them.  From Adequacy.org:

    Now if you followed the link above and read the article you may be thinking to yourself...
  • Is this what MS developers do when they should be concentrating on security?
  • How often do they audit their code?
  • What's to stop someone from inserting malicious code?
  • Is this why I pay so much for Windows and MS Office?
  • I know other non-MS software contains EEs but this is rediculous.
  • One more reason why peer review is better as EEs and malicious code can be removed quickly.
  • Is this why security patches takes so long to be released?
  • This is TrustWorthy Computing!?!
  • From technofile:

    Even more disturbing is the vision of Microsoft as the purveyor of foolishness. Already, the cloying "Easter eggs" that Microsoft hides away in its software -- surprise messages, sounds or images that show off the skill of the programmers but have no usefulness otherwise -- are forcing many users to question the seriousness of Microsoft's management.
       A company whose engineers can spend dozens or even hundreds of hours placing nonsensical "Easter eggs" in various programs would seem to have no excuse for releasing Windows with any bugs at all. Microsoft's priorities are upside down if "Easter egg" frills and other non-essential features are more important than getting the basic software to work right.

    From Agathering.net:

    "and some users might like to know exactly why the company allows such huge eggs to bloat already big applications even further"

    I've been involved in Easter Eggs in the past - the Exchange 5.0 POP3 and NNTP servers had easter eggs in them.  In our case, we actually followed the rules - we filed a bug in the database ("Exchange POP3 server doesn't have an Easter Egg"), we had the PM write up a spec for it, the test lead developed test cases for it.  We even contacted the legal department to determine how we should reference the contingent staff that were included in the Easter Egg. 

    But it didn't matter - we still shouldn't have done it.  Why?  Because it was utterly irresponsible.  We didn't tell the customers about it, and that was unforgivable, ESPECIALLY in a network server.  What would have happened if there had been a buffer overflow or other security bug in the Easter Egg code?  How could we POSSIBLY explain to our customers that the reason we allowed a worm to propagate on the internet was because of the vanity of our deveopers?  Why on EARTH would they trust us in the future? 

    Not to mention that we messed up.  Just like the NT 3.1 Easter Egg, we had a bug in our Easter Egg, and we would send the Easter Egg out in response to protocol elements other than the intended ones.  When I was discussing this topic with Raymond Chen, he pointed out that his real-world IMAP client hit this bug - and he was more than slightly upset at us for it.

     

    It's about trust.  It's about being professional.  Yeah, it's cool seeing your name up there in lights.  It's cool when developers get a chance to let loose and show their creativity.  But it's not cool when doing it costs us the trust of our customers.

     

    Thanks to Raymond, KC and Betsy for their spirited email discussion that inspired this post, and especially to Raymond for the awesome links (and the dirt on my broken Easter Egg).

     

    Edit: Fixed some html wierdness.

    Edit2: s/anacronym/anagram/

  • Larry Osterman's WebLog

    Early Easter Eggs

    • 65 Comments

    Jensen Harris's blog post today talked about an early Easter Egg he found in the Radio Shack TRS-80 Color Computer BASIC interpreter.

     

    What's not widely known is that there were Easter Eggs in MS-DOS.  Not many, but some did slip in.  The earliest one I know of was one in the MS-DOS "Recover" command.

    The "Recover" command was an "interesting" command.

    As it was explained to me, when Microsoft added support for hard disks (and added a hierarchical filesystem to the operating system), the powers that be were worried that people would "lose" their files (by forgetting where they put them).

    The "recover" command was an attempt to solve this.  Of course it "solved" the problem by using the "Take a chainsaw to carve the Sunday roast" technique.

    You see, the "Recover" command flattened your hard disk - it moved all the files from all the subdirectories on your hard disk into the root directory.  And it renamed them to be FILE0001.REC to FILE<n>.REC.

    If someone ever used it, their immediate reaction was "Why on earth did those idiots put such a useless tool in the OS, now I've got got to figure out which of these files is my file, and I need to put all my files back where they came from".  Fortunately Microsoft finally removed it from the OS in the MS-DOS 5.0 timeframe.

     

    Before it flattened your hard disk, it helpfully asked you if you wanted to continue (Y/N)?.

    Here's the Easter Egg: On MS-DOS 2.0 (only), if you hit "CTRL-R" at the Y/N prompt, it would print out the string "<developer email alias> helped with the new DOS, Microsoft Rules!"

    To my knowledge, nobody ever figured out how to get access to this particular easter egg, although I do remember Peter Norton writing a column about it in PC-WEEK (he found the text of the easter egg by running "strings" on the recover.com binary).

    Nowadays, adding an easter egg to a Microsoft OS is immediate grounds for termination, so it's highly unlikely you'll ever see another.

     

    Somewhat later:  I dug up the documentation for the "recover" command - the version of the documentation I found indicates that the tool was intended to recover files with bad sectors within it - apparently if you specified a filename, it would create a new file in the current directory that contained all the clusters from the bad file that were readable.  If you specified just a drive, it did the same thing to all the files on the drive - which had the effect of wiping your entire disk.  So the tool isn't TOTALLY stupid, but it still was pretty surprising to me when I stumbled onto it on my test machine one day.

     

  • Larry Osterman's WebLog

    Activate-As-Activator activates as activator

    • 3 Comments
    This particular problem burned my team somewhat the other day, so I figured writing a blog post about the issue was a good thing.

    When you activate a COM object (with CoCreateInstance), one of the things that COM has to do is to figure out the activation model for the newly instantiated COM object.

    Most of the time, the COM object is instantiated as an Activate-As-Activator object, however you can override this setting using the APPID registry value in your CLSID registration.

    In general there are four forms of activation - activate-as-activator, activate-as-user, activate-as-interactive-user and activate-as-service (you can see these in the dcomcnfg tool).

     

     

    These are controlled by various keys under the appid section.  If you don't have an appid, or your COM object is an InProcServer or LocalServer, you're typically going to be using activate-as-activator.

    There's a clever trick that we use (I don't know if it's intentional, we sort-of stumbled into it by accident).  For Vista, the actual audio engine runs out-of-proc from the audio service.  We do this primarily to isolate 3rd party code - if the 3rd party code crashes, we'll silently restart the audio engine - you'll get a horrendous glitch but that should be about the extent of the damage.

    When you're running a COM server, there's a call - CoRegisterClassObject that's used to register the class factory for all of your COM objects.  When COM tries to activate a COM object, before it looks in the registry, it looks to see if there's a server already registered for that class.  This is actually pretty cool - as long as we guaranteed that the engine was running, we could simply call CoCreateInstance and instantiate our internal COM objects.  There was an unexpected bonus that came along with that - since our classes didn't specify an APPID, COM instantiated the objects as Activate-As-Activator.  

    There was also an unexpected bonus - when you're activating an activate-as-activator  in turn checked to make sure that the only person activating the internal interfaces was running as our service (thus adding an extra level of defense-in-depth).

    Things were going great; we didn't need a class registry for our COM objects, things just worked fine.  Until Friday, when we were preparing for a routine RI.  All of a sudden, our calls to CoCreateInstance started failing with REGDB_E_CLASSNOTREG (Class not registered).   We sat there and scratched our heads for a while and gave up - we called up the COM wizards for help.

    After the COM developer debugged the problem for a bit, he figured out what had gone wrong.

    One of my changes had leaked an impersonation - I impersonated the client, and then forgot to revert to self.  I had run through all my unit tests and hadn't caught the problem.  The problem only showed up because the call to CoCreateInstance attempted to activate the COM object while impersonating, and the token of the user I was impersonating didn't match the token of the service, so the CoCreateInstance call failed.

    You could argue that the REGDB_E_CLASSNOTREG error is incorrect in this case, but...

    One slightly clever trick I picked up while trying to track down the impersonation:

    #if DBG
    void AssertNotImpersonating()
    {
        HANDLE threadHandle;
        if (OpenThreadToken(GetCurrentThread(), TOKEN_QUERY, TRUE, &threadHandle))
        {
            OutputDebugString(L"Called while impersonating, this will cause problems.\n");
            DebugBreak();
            CloseHandle(threadHandle);
        }
    }
    #endif
     

    I just sprinkled these throughout my code and was able to find the offending change quickly.

    Btw, please note that this only works if you're checking to ensure that you are NOT impersonating. If you invert the check (to assert that you ARE impersonating because the call to OpenThreadToken succeeded)), then you may have a problem: If you're impersonating at the Anonymous level of impersonation, then the call to OpenThreadToken will fail with ERROR_CANT_OPEN_ANONYMOUS.

  • Larry Osterman's WebLog

    Unintended consequences of adding APIs to the system

    • 31 Comments

    Yesterday, I wrote about a trick to reduce the number of bits in a number by one.

    It turns out that I've only ever had one opportunity to use this trick (although I ran into an instance of it when code reviewing some stuff the other day), back when I was writing the NT 3.1 network filesystem.

    One of the things that makes writing a network client so hard is that you need to map legacy versions of your network protocol to the semantics that your customers require.  In this case, I needed to map between the "how much disk space is free on the drive" SMB request and the NT filesystem query that retrieves the amount of disk space on the drive.

    It turns out that the legacy servers returned this information in terms of bytes per sector, sectors per cluster and clusters per disk (because that was how MS-DOS maintained that information).

    NT maintained the information in a different form, and the conversion between the two forms required that I divide the interim results by bytes per sector.

    Well, if the bytes per sector was a power of two (which it almost always was), I could do the division trivially (by shifting the interim result right by the base2 logarithm of the sector size). 

    Unfortunately, if the bytes per sector was NOT a power of two, I was stuck with a division.  Since the interim value was a 64bit number, that meant that I had to do a 64bit by 32bit division.  At the time I was writing the code, the compilers didn't have support for 64bit arithmetic, for 64bit math, we had a number of internal "RTL" functions that would do the math.  We had functions for addition, subtraction and multiplication, but nobody had ever written a version that did division.

    I'd never done numeric processing before, but I needed the function, so...  I went to Dave Cutler (since he knows everything) and asked him how to do the math, he told me and I trotted off to implement his algorithm.

    Since the code was only used in a special case that was likely never to occur outside of the test labs, I didn't particularly spend a lot of time optimizing the function - I faithfully coded the algorithm in C - two versions, one which did a 64x64 division, another that did a 64x32 division.  I could have written an assembly language version of the 64x32 version (since the processor supported 64x32 division natively), but we were REALLY concerned about portability and since it wasn't going to be called that often, I went with the more portable solution (and I'd been told that if I ever wrote stuff in assembly language, I'd have my thumbs slowly roasted over a red hot hibachi).

    So I wrote the API, wrote enough unit tests to confirm to myself that it worked, checked it in as a new RTL API and went away satisfied that I'd done my first piece of numerical processing.  I was SO happy :)

    At this point, anyone reading this post who was on the NT team at the time is starting to see where this is going.

     

    About three years later (long after I'd left NT and moved onto Exchange), I heard a story about something that happened some point after I'd left the team.  You see, Dave spent a fair amount of time during the NT 3.5 to NT 4 period doing analysis of the bottlenecks in the system and looking for ways of making the system faster.

    At some point, his investigations brought him to (you guessed it) my little division functions.  It turns out that one of the core components in the system (I think it was GDI, but I'm not sure) decided that they needed to divide a 64bit numbers by a 32bit number. 

    Because the compiler didn't have native support for 64bit arithmetic, they went looking through the same RTL functions I did, and they discovered my 64x32 division routine.  So they started using the routine.

    In their innermost loop.

    All of a sudden, the silly routine I'd written to solve a problem that would never be seen in real life all of a sudden became a part of the inner loop of a performance critical function inside the OS.

     

    When Cutler discovered this bottleneck, he looked at the code in my little routine and he went through the roof.  I understand he went through the roof and started cursing my name repeatedly before he rewrote it in assembly language.

     

    So the moral of the story is: When you're writing code that's going into a system library, make darned sure that it's written to be as performant as humanly possible, because you never know if someone's going to find your one-off piece of code and put it in their innermost loop.

  • Larry Osterman's WebLog

    Another of my favorite tricks - reducing the number of bits in a number by 1

    • 39 Comments

    One of the classic (and thus no longer asked) Microsoft interview questions is "How quickly can you count the bits in a 16 (or 32) bit integer?".

    You get a varied number of responses to this one, from brute force to lookup tables.

    One of my favorite tricks for this is:

       x = x & (x - 1)

    This reduces the number of bits in a number by one.  I'm not sure exactly why it works, and it only works for 2s complement numbers, but it DOES work.

    So using this trick, the bitcount question is:

    {
       bitcount = 0;
       while (value != 0)
       {
         bitcount += 1;
         value = value & (value - 1);
       }
       return bitcount;
    }

    It's a silly trick, but clever.

     

    Btw, being geeks, a number of people over here came up with the fastest known version (in software).  It involves adding the bits in parallel - you can count the bits in 32bit integer in a constant 34 cycles on a 386 machine.

    To do that, you first split the number into odd and even bits thus creating 32 1 bit numbers.

    Next, you shift the even bits right one bit and add the values together to get a collection of 16 2 bit numbers.

    Now split the number into groups of 2 bits by again masking by 0b0011001100110011 and 0b1100110011001100, shift the second value right by two bits and add the values to get a collection of 8 4 bit numbers.

    Now split the number into groups of 4 bits by again masking by 0b0000111100001111 and 0b1111000011110000, shift the second value right by four bits and add the values to get a collection of 4 8 bit numbers.

    Now split the number into groups of 8 bits by again masking by 0b0000000011111111 and 0b1111111100000000, shift the second value right by eight bits and add the values to get a collection of 2 16 bit numbers.

    Finally, split the number into groups of 16 bits by taking the low sixteen bits and adding them to the high 16 bits, you now have one 32 bit integer that contains the number of bits that were on in the original value.

     

  • Larry Osterman's WebLog

    It's on the whiteboard

    • 14 Comments

    Way back when, when we were first shipping NT 3.1, checking files into the source tree was pretty easy.  You made your changes and checked them in. Not a big deal, since there were only 20 or so people working on the code base - the chances of collision were relatively small, and the codebase was pretty managible.  There was a small team of people who had the job of doing nightly builds, it was their responsibility to ensure that a build was done every day, and that the build worked and passed BVTs (the team was something like 5 people, if I recall).

    At some point, a number of groups joined the core NT team, and the NT team grew to a couple of hundred developers.  Not surprisingly, the system that had worked for the 20 or so people didn't scale to the hundreds of people who were now using the system.  It got so bad that we often went for days at a time without being able to have a good build.

    We tried community shame (if I had a scanner here at work, I'd scan in the picture of me from back in those days wearing goat horns), it didn't work (I have no shame).  We tried staged checkins (each team gets a dedicated hour to check in).  We tried darned near everything, but the problem was that our old system simply didn't scale to the size of the group. 

    Eventually things got so bad that Dave Cutler ended up moving into the NT build lab to directly supervise the builds.  It was a varient of the "community shame" solution, but instead of being forced to wear a silly costume, you had to explain why you screwed up to Dave directly, and it was FAR more effective (there's nothing like being grilled by Dave Cutler to instill fear into a developer).

    In order to manage the volume of changes, Dave instituted the "Whiteboard".  Basically he got Microsoft to buy a 4 foot by 12 foot whiteboard, and had them mount it vertically in the build lab across from where he sat.  When you had a change ready to check in, you went to the whiteboard and wrote the your name, the bug #, the module being changed, and a contact number.  Dave would then periodically run down the board and call individuals to get them to check in their changes.  The cool thing about this mechanism was that Dave could control the build process - he could do sanity builds after individuals (like me) who had a propensity of breaking the build, he could batch changes from the same group together, etc.

    It also provided a dramatic visual representation of the state of NT - when the whiteboard was full, the product had lots of bugs, when it was clear, we were close to being done.  And when it was empty, we had shipped the product.

     

    Of course, the whiteboard didn't really scale, even to a project the size of NT 3.1.  And today, Vista is vastly more complicated - there are several thousand developers contributing code into a a bazillion binaries composed of a gajillion source files (I don't know how many, but there's a lot of them).  There's no way that the whiteboard could concievably scale today.  Instead, we have a main build lab (which produces the final bits of the product) and a series of "virtual build labs", each of which is responsible for aggregating changes from a set of Windows developers.  Its far more scalable than the old system, and significantly more flexible (at a minimum, it doesn't require that a Senior Distinguished Engineer spend all his time making sure that the build completes successfully).

     

  • Larry Osterman's WebLog

    Looking under rocks...

    • 19 Comments

    I've really been enjoying Jensen Harris's blog recently, he's been talking about the UI design that went into Office 12.

    Today's post  (on turning over rocks) is no different, reading it brought a smile to my face, mostly because of this paragraph:

    I don't know why we say "under rocks."  Maybe I made it up, maybe I heard it somewhere, who knows.  The picture I get in my head is an insect-eating animal crossing the land, turning over rocks to look for meals.  Occasionally, a rock will be hiding a juicy insect.  Most times, however, there's nothing under the rock.  As a result, the animal spends most of his day looking under rocks.

    You see, when I think of looking under rocks, I think of an event that happened way back when I was 5 years old or so.  My friend Danny Dexter and I were out hunting for worms (it's a 5 year old thing) on a wooded hillside near our homes.  We were lifting up rocks, clawing through the dirt underneath the rocks looking to see if there were any worms, and then putting the rocks back.

    We'd been at this for a while, and I pushed up a HUGE rock (ok, huge for a 5 year old).  I couldn't hold the entire weight of the rock, so I just tipped it back.  Underneath the rock was the motherload of worms - there had to be a bazillion of them (ok, this was 35+ years ago, forgive me for my poor memory).  So with my left hand, I'm scrabbling in the dirt trying to find more worms.

    And I took my right hand off the rock, and started waving "Hey Danny!".

    <at this point, the storyteller pauses for dramatic effect as the listeners watch him pretending to paw the ground with his left hand while waving with his right hand (previously used to prop up the rock)>

    Crunch!  The rock fell down, and Larry's left hand was somewhat smushed underneath it.

    6 stitches later, I had a souvenir of the day, it's a 2 inch scar running lengthwise across my left palm.  No loss of functionality, just a white line.  But a good story anyway :)

    For the next 15 years or so, I used the scar to tell my left hand from my right (I'm slow that way).  To this day, when giving directions, I have to fight the urge to hold up my left hand to confirm which side is left.

     

    Anyway, check out Jensen's blog, it's absolutely worth it.

     

  • Larry Osterman's WebLog

    Does __fastcall make a difference for C++ classes?

    • 15 Comments
    We're running through a routine round of code reviews of the audio engine, and I noticed the following code (obscured):

    HRESULT __fastcall CSomeClass::SomeMethod(SomeParameters);  

    I looked at it a couple of times, because it seemed like it was wrong.  The thing that caught my eye was the "__fastcall" declaration.  __fastcall  is a Microsoft C++ extension that allows the compiler to put the first 2 DWORD parameters to the routine into the ECX and EDX registers (obviously it's x32 only).

    But when compiling C++ code, the default calling convention is "thiscall", and in the thiscall convention, the "this" pointer is passed in the ECX register, which seems to collide with the __fastcall declaration.

    So does it make a difference?  I could have left a code review comment and made the person who owned the code run through the exercise, but I figured why not figure out the answer myself?  And, to be honest, I found the path to the answer almost more interesting than the answer itself.

    As I usually do in these cases, I wrote a tiny little test application to test it out:

    class fctest
    {
       
    int _member;
    public:
        fctest::fctest(
    void);
        fctest::~fctest(
    void);
        int __fastcall fctest::FastcallFunction(int *param1, int *param2)
        {
           
    return *param1 * *param2;
        }

        int fctest::ThiscallFunction(int *param1, int *param2)
        {
           
    return *param1 * *param2;
        }
    };

    int _tmain(int argc, _TCHAR* argv[])
    {
        fctest test;
       
    int param1, param2;
       
    int result;
        result = test.FastcallFunction(&param1, &param2);
        result = test.ThiscallFunction(&param1, &param2);
       
    return 0;
    }

     

    I compiled it for "Retail", and then I looked at the generated output.  Somewhat to my surprise, the code generated was:

    main:
        xor eax, eax
        ret

    Yup, the compiler had optimized out my entire program.  Crud, back to the drawing board.

    Try #2:

    int _tmain(int argc, _TCHAR* argv[])
    {
        fctest test;
       
    int param1, param2;
       
    int result;
        result = test.FastcallFunction(&param1, &param2);
        printf("%d: %d: %d", param1, param2, result);
        result = test.ThiscallFunction(&param1, &param2);
        printf("%d: %d: %d", param1, param2, result);
       
    return 0;
    }

    This one was somewhat better:

    main:
        mov eax, [sp]
        imul eax, [sp+4]
        <call to printf #1>
        <call to printf #2>
        xor eax, eax
        ret

    Hmm, that wasn't much of an improvement.  The compiler realized that FastcallFunction and ThiscallFunction did the same thing and not only did it inline the call, but it optimized out the 2nd call.

    Try #3:

    int _tmain(int argc, _TCHAR* argv[])
    {
        fctest test;
       
    int param1, param2;
       
    int result;
        param1 = rand();
        param2 = rand();
        result = test.FastcallFunction(&param1, &param2);
        printf("%d: %d: %d", param1, param2, result);
        param1 = rand();
        param2 = rand();
        result = test.ThiscallFunction(&param1, &param2);
        printf("%d: %d: %d", param1, param2, result);
       
    return 0;
    }

     

    Try #3's code:

    main:
        call rand
        mov [sp], eax
        call rand
        mov [sp], eax
        mov eax, [sp]
        imul eax, [sp+4]
        <call to printf #1>
        call rand
        mov [sp], eax
        call rand
        mov [sp], eax
        mov eax, [sp]
        imul eax, [sp+4]
        <call to printf #2>
        xor eax, eax
        ret

    Much better, now at least both functions are inlined.  But the stupid function is STILL inlined, I haven't learned anything yet.

    Try #4: I moved fctest into its own source file (I'm not going to show the source code for this one).

    The code for this one finally got it right:

                 param1 = rand();
    00401029 call rand (401131h)
    0040102E mov dword ptr [esp+4],eax
                
    param2 = rand();
    00401032 call rand (401131h)
    00401037 mov dword ptr [esp],eax
                
    result = test.FastcallFunction(&param1, &param2);
    0040103A lea eax,[esp]
    0040103D push eax
    0040103E lea edx,[esp+8]
    00401042 lea ecx,[esp+0Ch]
    00401046 call fctest::FastcallFunction (4010E0h)
                printf("%d: %d: %d", param1, param2, result);
    0040104B mov ecx,dword ptr [esp]
                param1 = rand();
    00401062 call rand (401131h)
    00401067 mov dword ptr [esp+4],eax
                param2 = rand();
    0040106B call rand (401131h)
    00401070 mov dword ptr [esp],eax
                result = test.ThiscallFunction(&param1, &param2);
    00401073 lea eax,[esp]
    00401076 push eax
    00401077 lea ecx,[esp+8]
    0040107B push ecx
    0040107C lea ecx,[esp+10h]
    00401080 call fctest::ThiscallFunction (4010F0h)

    So what's in all this gobbeldygook?

    Well, the relevant parts are the instructions from 0x4013a to 0x40146 and 0x401073 to 40107c.  Side by Side, they are:

    0040103A lea eax,[esp]
    0040103D push eax
    0040103E lea edx,[esp+8]
    00401042 lea ecx,[esp+0Ch]
    00401046 call fctest::FastcallFunction (4010E0h)
    00401073 lea eax,[esp]
    00401076 push eax
    00401077 lea ecx,[esp+8]
    0040107B push ecx
    0040107C lea ecx,[esp+10h]
    00401080 call fctest::ThiscallFunction (4010F0h)

    Note that on both functions, the ECX register is loaded with the address of "test".  But in the fastcall function, the 1st parameter is loaded into the EDX register - in the thiscall function, it's pushed onto the stack.

    So yes, __fastcall makes a difference for C++ classes.  Not as much as it does for C functions, but it DOES make a difference.

     

  • Larry Osterman's WebLog

    Setting volume on audio CDs with MCI

    • 10 Comments

    So I got an email from a reader yesterday asking:

    How do you set the volume of an audio stream coming from a CD? I found an old piece of code that used mciSendCommand with some parameters to do so, but my compiler cannot find the "MCI_SET_VOLUME" constant. Has this been completely depreciated? If so, how can I control the volume of CD output?

    It surprised me, I thought you could set the volume on audio playback, so I dug a bit deeper.

    My first thought was the MCI "SET" verb - MCI_SET_AUDIO allows you to control audio channels during playback.

    The reader responded that the MCI_SET_PARMS doesn't have a lVolume member like you'd think it should.  So I dug deeper, and discovered that you can't set the volume of CD playback via MCI.  Your only option (using MCI only) is to mute either the left, right, or both channels (using MCI_SET_AUDIO with the MCI_SET_AUDIO_ALL/MCI_SET_AUDIO_LEFT/MCI_SET_AUDIO_RIGHT dwAudio fields, and a MCI_SET_ON or MCI_SET_OFF flag.

    Very strange, I don't know why this is.  But that's the way it's been since at least windows XP (the oldest source I have available).

     

    If you REALLY must control the output volume, then you've only got a couple of options.  First off, you can use the mixer APIs and search for a CD source line, the volume control on the CD source will control the volume of the CD output.  I also believe that that the aux APIs can be used to control the CD audio volume (auxGetDevCaps will let you know if an aux device is a CD audio device).

    Having said that, there may be compatibility issues using these techniques when run on Windows Vista.

    The good news is that for new code, it's actually pretty easy to write code to read the data from the CD directly and play it back, I wrote a sample version of such code here.  And if you're using the wave APIs to render audio, the wave audio playback APIs should work perfectly.

     

  • Larry Osterman's WebLog

    Sorry for the lack of technical stuff recently...

    • 2 Comments

    I've been swamped with work, and all the reviewers for the Vista audio posts haven't gotten back to me (you know who you are :)) so I've lagged behind (yes, the Vista audio stuff is being reviewed by others, it's important that the posts be accurate).

    So instead of doing a series on Vista audio, I'll be posting the articles as they come back from review, so it won't be concentrated Vista audio posts, but other tech stuff that I've been putting on hold in favor of the Vista audio stuff.

     

    The Vista audio stuff will be forthcoming, it's just going to take a bit more time :(

     

  • Larry Osterman's WebLog

    I don't know if I should be honored or embarassed...

    • 3 Comments

    There's this guy, "Jamie" over on Channel9 who's quite skilled with photoshop.

    One of his ongoing themes is something he calls "Channel9 Park" - it's Microsoft related stories with a South Park theme.

     

    Apparently I showed up in todays episode.  As I said, I'm not sure if I should be honored or embarassed.

    I'm also not sure I understand the comic, but...

    Oh, and my beard's much shorter nowadays :)

     

Page 1 of 1 (16 items)