From Eric Sink, via Dana Epp:
"My Life as a Code Economist"
And Dana's original article:
"The cost of fixing bugs..."
Yeah, I said I was on vacation, but this is SO depressing. I just saw this on the front page of MSNBC.COM.
It describes a new varient of Sober, which is forwarded as an executable contained in a .ZIP file.
I wrote about the issue back in July, but it's still depressing/annoying to see things like this still going around.
Anyway, back to your regularly scheduled turkey-day.
As you can probably tell from the lack of posts to this blog, I'm feeling a bit burned out putting the finishing touches on my parts of Vista Beta2, so I'm taking the next week off.
So no new posts until at least the 28th.
Everyone in the US, have a happy Thanksgiving, and I'll see y'all in a week.
At some point, the conversation drifted around to our motivations for working on NT, and Tom made a comment that's stuck with me for the 15 years since he made it.
"I work on NT because it's an opportunity to work on the last new PC operating system to be written in my lifetime."
It's true that many of the low level concepts embodied in NT originated in the operating systems that Dave Cutler and the core NT team developed when they were at DEC, but the entire NT operating system was written from the ground up - with the exception of the command interpreter, much of the code in USER, the TCP/IP stack (licensed from Spider systems) and a couple of the networking utilities, every bit of code in NT 3.1 was brand spanking new. And there were plans in place to replace the TCP/IP stack with an in-house TCP/IP stack (delivered with NT 3.5), so even that was written from the bottom up.
When I relate Toms comment to others, they invariably respond "But what about [Linux|OSX|Be]". Taken in turn:
Linux was started in 1991, after the fact, but it's not a complete OS - Linux is the kernel, but the other parts of the OS date from other projects started long before Linux was started. The kernel is new, but the entire OS isn't - all the user mode components come from previous operating systems, as does the networking stack.
OSX is based on FreeBSD, which was in turn based on 386BSD - large parts of the OS have been reengineered, but again, it wasn't a bottom-up attempt to write an entire OS.
Be's OS might have qualified as a new OS, since it was written from the bottom up, but unfortunately it didn't succeed in the marketplace.
There have been a number of original OS's built for various small devices since then (Palm, Symbian, WinCE, etc), but none for PC's.
In reality, this isn't really surprising, it takes a HUGE amount of effort to launch a complete operating system - hundreds of millions of dollars just in direct development costs. I still remember when we had the first big scheduling meeting when we laid out the list of things that we had to have for NT - many of the developers in the room were aghast at the number of components that simply HAD to be present to be considered a complete operating system: 16 bit application compatibility (which was a real issue for our non x86 platforms), printer drivers, video drivers, printing UI, multimedia hardware support, networking support, filesystem utilities, the list went on and on.
Nowadays, the list is even longer - for instance, back in 1989 when that scheduling meeting happened, you didn't need a web browser, because the web hadn't been invented yet (the first web page was written in 1990), but today, clearly you'd have had to have a web browser on the list. Similarly, today you need to have camera support, digital media player support, instant messenger support, etc.
My suspicion is that Tom's statement will continue to hold true - the amount of effort required to build an entire operating system from the bottom up is daunting, and it's unlikely that anyone is likely to embark on such an effort in the foreseeable future.
It turns out that my neighbor works in the same building as I do, she's the group product manager for IE (go figure that one). I ran into her the other day, and she pointed me to this video she did for "hive.net".
I've got to say that I was really impressed. The content is primarily marketing stuff (not technical), but Margaret is really funny, and she's got a great camera presence. She also does an awesome job of presenting lots of the new features in IE7.
Very, very nice, and utterly worth watching - it's a joy to watch her present this. Unfortunately, it's a bit hard to see what she's showing on the monitor, that's a limitation of filming PCs with a camcorder :(.
Apparently it was on Conan last night, but...
I normally don't like Triumph the comic insult dog, but for some reason, I thought that this was hysterical (maybe because I got too little sleep last night or something, I don't know).
Anyone who's been at Microsoft for long enough (long enough to use DOS on a day-to-day basis) remembers the deadly "beeping death".
The "beeping death" was an artifact of the MS-NET product that we deployed for networking here at Microsoft, and I was the developer responsible for the "beeping death".
What was the beeping death? Well, it occurred because a confluence of about whole lot of different parts of the networking system.
First, you need to understand how connection oriented network protocols work (this is a VERY rough description). When the client sends a request to the server, the server acknowledges receipt of the package with a packet called an "ACK". In addition, on a connection oriented protocol, the connection itself is kept alive by the client periodically sending a message (called a "keep-alive") to the server - that way, even if there's no network traffic, the client will eventually discover if the server has gone away.
Secondly, the MS-NET product (and the DOS Lan Manager product after it) used the NetBIOS API layer to talk to the network adapter. But in reality, it didn't. The MS-NET product instead talked to an abstract networking API layer called the "session" layer, which was a part of the MS-NET product. From an API standpoint, it was extremely similar to the NetBIOS API layer, but it wasn't quite the same.
Third, there were two different implementations of the session layer. One (session.exe) was a sample version that was shipped with the OEM kit for MS-NET. The other was called minses.exe. Minses.exe provided a minimal session layer that was intended to interface with NetBIOS. So it functioned as a mapping layer between the MS-NET components and the actual networking stack.
Now one of the cool features of minses was that on synchronous networking calls, the minses would beep the PC speaker while the call was outstanding. That would let the user know that the system was still thinking about their request, and it hadn't forgotten them.
Fourth, the Microsoft corporate network at the time was (and still is, to my knowledge) the largest, most complicated corporate network on the planet. We have branch offices in hundreds of countries, there are hundreds of thousands of computers on the network, it's REALLY big network (Raymond tells this story about the network back in the 1990s). It's a REALLY big, really complicated network. That means that there are a bazillion failure points on the network, which means that connectivity often went down.
And finally, the networking solution we used back then was based on Ungermann-Bass smart network cards. These cards were pretty cool actually - when you started the system, the OS downloaded the entire network stack onto the card, which mean that system memory didn't get consumed by the networking stack. With this fifth piece, the networking guys reading this should start saying "Uh oh"...
Now that I've set the stage for the confluence of features, lets see what happens when this system gets deployed in real life..
In the normal case, everything works fine - you never ever hear the beep, because responses come back before the beep comes out. But that's the most uninteresting problem (for networking environments, the normal case is usually profoundly uninteresting - it's when things start failing that things get exciting)...
And the "beeping death" scenario was no different - it gets interesting when you start looking at the ways that things can fail.
Lets consider some of the failure modes:
1) Connectivity fails on an intervening network node between the client and the server.
In that case, the client hangs waiting on the network to time out. This could take several seconds, sometimes even as much as a minute. Bad, but not the end of the world, because the timeouts within the transport detect the connectivity problem and fail the request.
2) The client crashes (this IS MS-DOS, we're talking about).
In this case, the connection is held alive (remember - the actual network transport is running on the UB card, not taking up system memory), and that ties up some resources on the server but it's still not the end of the world (from a clients perspective)
3) The server computer is really busy.
In this case, the client waits until the server comes back to it. That may take time and can be really annoying.
4) The server crashes, or otherwise freezes (breaks into the kernel debugger, etc).
In this case, the server disappears. If the timing of the request was correct, the servers crash tears down the connections and the clients get networking failures. If, on the other hand, you're unlucky, the network card might have received the client's request and handed it to the server, but the server hadn't responded to the client. In that case, there were no outstanding network requests for the client. Because the transport is sitting running entirely on the network adapter, it has no way of knowing that the host operating system is dead. The transport just sits there, quietly acknowledging the keep-alive It sits on the network card until the operating system is rebooted. It can get even more heinous when the server process and is restarted - in that case, the card would sometimes "forget" existing connections until the operating system was rebooted (or power was recycled on the server).
If you happened to be one of the poor clients stuck in this state, they sat there blocked on a synchronous network receive waiting for the frozen server to respond to their request. Since the server process was gone (or the machine was in the debugger, or...), the client never had an opportunity to detect the failure. And, since DOS was a single threaded operating system, and the networking requests were executed in the kernel, the user had no choice but to reboot their client.
I got indescribable amounts of flack for the beeping death, because it seemed that every time any server crashed anywhere at Microsoft, some set of clients would start beeping forever... Fortunately, many of the changes I made for DOS Lan Manager 2.0 removed the beeping death (it allowed the client to detect a hung server and tear down the connection to the server even if the underlying network claimed that things were just fine).
Networking can be fun :)
It turns out that Windows will still generate calls to beep.sys under certain circumstances.
The most common place you're likely to hear beep.sys is if there's a problem with your machines audio solution (either it's malfunctioning, the drivers aren't working, or its been disabled or removed). In that case, then all calls to PlaySound will result in calls to Beep() (assuming that the system is configured to beep on playsound).
There's one other circumstance in which the system will play beeps, and that's for accessibility sounds. Accessibility events are typically associated with audio events - these allow a visually impaired user to recognize that an event of significance has occurred.
The PlaySound beep behavior is actually gated on a system parameter - you can call the SystemParametersInfo API to retrieve SPI_GETBEEP to retrieve whether the "warning beeper" is on. You can also disable the "warning beeper" by calling SystemParametersInfo with the SPI_SETBEEP flag.
Note that disabling the "warning beeper" has no affect on the accessibility sounds. those are generated regardless of the warning beep setting.
It's one of the oldest Windows API, dating back to Windows 1.0. It's also one of the few audio APIs that my team doesn't own. The Beep API actually has its own dedicated driver (beep.sys). The reason for this is that the Beep() API works totally differently from any other audio API in the system.
Back when IBM built the first IBM PCs, they realized that they needed to have the ability to do SOME level of audio, even if it wasn't particularly high quality. So they built a speaker into the original PC hardware.
But how do you drive the speaker? It turns out that the original PC hardware used an 8253 programmable interval timer to control the system hardware timer. The 8253 was a pretty cool little chip - it would operate in 5 different modes - one shot timer, interrupt on terminal count, rate generator, square wave generator, software strobe or hardware strobe. It also contained three independent counters - counter 0 was used by the operating system, counter 1 was reserved for the hardware. The third counter, counter 2 was special. The IBM hardware engineers tied the OUT2 line from the 8253 to the speaker line, and they programmed the timer to operate in square wave generation mode.
What that means is that whenever the 2nd counter of the 8253 counted to 0, it would toggle the output of the OUT2 line from the 8253. This gave the PC a primitive way of generating very simple tones.
The original Windows Beep() API simply fiddled the controls on the 8253 to cause it to generate a square wave with the appropriate frequency, and that's what Beep.sys continues to do. Legacy APIs can be hard to remove sometimes :)
Nowadays, the internal PC speaker is often also connected to the PCs audio solution, that allows the PC to have sound even when there are no external speakers connected to the machine.
In addition to the simple beep, some very clever people figured out how they could use the 8253 to generate honest to goodness audio, I'm not sure how they succeeded in doing it but I remember someone had a PC speaker based sound driver for DOS available at one point - it totally killed your PCs performance but it DID play something better than BEEEEEEP.
Edit: s/interrupt conroller/interval timer/
Edit2: fixed description of channel 1 (in case someone comes along later and decides to depend on my error).
Michael Gartenberg pointed out this cool illusion. It's a circle of pink dots with a rotating highlight. If you stare at it long enough, the pink highlight turns green, and if you keep on looking at it, the circle of pink dots disappears.
Very, very freaky.