June, 2005

  • The Old New Thing

    Beware of roaming user profiles


    One of the less-known features of Windows is the roaming user profile. I know that this is not well-known because I often see suggestions that fail to take the roaming user profile scenario into account. Indeed, if your program behaves badly enough, you can cause data loss. (More on this later.)

    What is a roaming user profile?

    Well, your user profile is the collection of things that reside under your %USERPROFILE% directory. (This is not quite true, but it's a good enough approximation for the purpose of this discussion. An important exception will be noted next time.) Your per-user registry is kept in %USERPROFILE%\ntuser.dat, so your per-user registry is part of your user profile.

    In highly managed environments (corporations), system administrators can set up user profiles on a centralized server, so that users log onto any machine and have available their files and settings. This is accomplished by copying the user profile from the server when the user logs on and copying it back to the server when the user logs off. (Of course, there is also caching involved to save time if the user logs back onto the same machine.)

    What does this mean for you, the programmer?

    For one thing, it means that the path to the user's profile can change from one logon session to the next. If the user runs your program from Computer A, their user profile directory might be C:\Documents and Settings\Fred, but when they log off from Computer A and log on to Computer B, the directory to their user profile might change to C:\WINNT\Profiles\Fred. In particular, that file that used to be at C:\Documents and Settings\Fred\My Documents\Proposal.txt has moved to C:\WINNT\Profiles\Fred\My Documents\Proposal.txt. If your program has a feature where it offers a list of recently-used files (or auto-opens the most recently used file), you may find that the file no longer exists at its old location. The solution is to use profile-relative paths, or even better, shell virtual folder-relative paths (e.g., recording the path relative to CSIDL_MYDOCUMENTS), so that when the profile roams to a machine with a different user profile path, your program can still find its files.

    For another thing, you cannot just cruise through the HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\ProfileList registry key expecting to find all the user profiles and possibly even modify them, because the copy of the user profile on the local computer might not be the authoritative one. If the profile is a cached roaming profile, then any changes you make will either (1) be lost when the user roams back to the computer after using another computer, or (2) cause the local profile to be considered newer than the master copy on the server, causing the changes the user made to the copy on the server to be lost! (Which of the two bad scenarios you find yourself in depends on the time you change the cached profile and the time the target user logs off that other computer.)

    Another consequence of roaming user profiles is that your program can effectively see itself changing versions constantly. If Computer A has version 1.0 of your program and Computer B has version 2.0, then as the profile roams between the two computers, both versions 1.0 and 2.0 will be operating on the user profile in turn. If versions 1.0 and 2.0 use the same registry keys to record their settings, then your registry formats had better be both upward- and downward-compatible. This is a particularly painful requirement for operating system components, which consequently need to maintain bidirectional registry format compatibility with systems as old as Windows NT 4. (Windows NT 3.51 had a different model for roaming user profiles.)

    Yet another consequence of roaming user profiles applies to services. Prior to Windows XP, if a service holds a registry key open after the user logged off, then the registry hive cannot be unloaded and consequently (1) consumes memory for that profile even though the user is no longer logged on, and (2) prevents the user's local registry changes from being copied back to the server. This "hive leakage" problem was so rampant that in Windows XP, the profile unload code takes a more aggressive stance against services that hold keys open too long. You can read more about the changes to registry hive roaming in the Resource Kit article linked at the top of this entry.

  • The Old New Thing

    A simple story in pictures


    Professionals at work is the title of this series of unattributed pictures. (Via memepool Nov 26, 2004.) I think they speak for themselves.

    (I apologize in advance for the lowbrow advertisements on the page.)

  • The Old New Thing

    Why isn't Windows Setup just a disk reimage plus a registry dump?


    Why can't all the registry key needed by Windows Setup be precomputed and splorted at one go? One reason is that Windows Setup can be customized via a so-called "unattend file", which means that a lot of the work cannot be precalculated. Some settings also have cascade effects on other settings.

    Also, the model for component registration is to make each component responsible for its own registration, which can in turn be context-sensitive: "If the system has configuration x, then do y else z". Internet Explorer's component registration for example contains many conditional sections based on the installation configuration specified in the unattend file and the even fancier Winbom.ini file used by Sysprep-based deployment (which is used by computer manufacturers and large corporations).

    Making each component responsible for its own registration reduces entanglements between components but also means that "global optimizations" such as precalculating the registry cannot be done. Engineering is about tradeoffs and this is one of them: Maintainability vs. speed.

    (Of course, there are sections of the registry that are not context-sensitive, and those to some degree have already been gathered together for block copying onto the machine during Windows Setup. Look at the HIVExxx.INF files on your Windows XP CD.)

    That said, it appears that Longhorn setup is moving towards the "disk image" model. I wish them luck.

  • The Old New Thing

    Drink at work: The blog of a cartoonist


    Francesco ("Ces") Marciuliano is the author of the comic strip Sally Forth. In his blog, Drink at Work he expounds on whatever tickles his fancy. (Warning: Includes strong language, adult situations, and political views.) Sure, the stories on writing a cartoon are fun, especially when he responds to feedback from readers, and he also has a long series of articles titled Comic Strip Writing 101 (example: Comic Strip Writing 101: It's Not All Pabst and Vicodin), but that's not the best part. No, the best part is the conversations with his father. Here's an except from Conversation number two:

    Dad: Marciulianos live much longer than average folks! Look at your Grandpa! He would still be alive today if he hadn't died in that hospital.

    Ces: What?

    Dad: Y'know, from that spill he took...when he had to go to the hospital. I bet if he didn't fall he would still be around today.

    Ces: At age 102?

    Dad: See? That's what I'm talking about. Marciulianos live a long time.

  • The Old New Thing

    Why do Microsoft code samples tend to use ZeroMemory instead of { 0 }?


    If you go browsing around MSDN, you'll find that code samples tend to call ZeroMemory explicitly rather than using "= { 0 }" notation. Why is that?

    To make it clearer for people who are used to other programming languages.

    Like it or not, a significant percentage of people who write programs for Windows do it in languages other than C and C++. Although those developers may have a basic understanding of C and C++, they don't have all the language subtleties memorized.

    Compare the situation of speaking in English to a group of people where not everyone speaks the language fluently. If you're considerate of your audience, you're going to avoid the more esoteric grammatical constructions, the rare words, and the obscure idioms. Instead of saying, "Were it to rain, we will see that umbrellas be available," you would use the simpler "If it rains, then we will make sure that umbrellas are available," thereby avoiding the grammatical complexity of the implied conditional by inversion ("if"), the subjunctive of condition ("were"), the implied conclusion ("then"), and the subjunction of intention ("be").

    Heck, even people who claim to know C and C++ don't have all the language subtleties memorized. Some of them have false impressions of what " = { 0 }" does. And who among us really has C/C++'s bizarre operator precedence rules committed to memory?

    Consequently, MSDN samples tend to use ZeroMemory to make it blindingly obvious what is being set to zero. One of the things we've learned over the years is that many people just copy/paste sample code without understanding it. If there are little things like ZeroMemory that can be done to make the intent of sample code clearer and reduce translation errors, then that's a good thing.

    If you prefer " = { 0 }", then go ahead and use it, secure in the knowledge that thousands of programmers aren't going to read your code and try to translate it into Visual Basic because that's the only language they know. But MSDN doesn't have that luxury.

  • The Old New Thing

    Why are there broadcast-based mechanisms in Windows?


    Many Windows information mechanisms are based on message broadcasts, among them DDE, WM_FONTCHANGE, and changes in system settings. Why do these mechanisms use broadcasts, when we know that broadcasts can result in the system grinding to a halt due to windows that have stopped processing messages?

    Because in 16-bit Windows, you didn't have this problem.

    Recall that 16-bit Windows was co-operatively multi-tasking. When a program received control of the CPU, it could do anything it wanted, knowing that no other programs could run until it explicitly yielded control by calling a function such as GetMessage or PeekMessage. The downside of this, of course, was that a single hung program caused the entire system to hang, because it wasn't releasing the CPU.

    The upside, however, was that if your program was running, then you knew, a priori, that there were no hung programs in the system. How do you know that? Because if there were a hung program, it would be running and not you.

    If there's only one thing, and you have it, then you know that nobody else is hogging it.

    Therefore, broadcasting messages was completely safe in 16-bit Windows. You didn't have to worry about non-responsive programs because you had proof that there weren't any.

    Of course, when the switch to pre-emptive multi-tasking occurred, this assumption no longer applied, but by then it was too late. The broadcast-based model was already in use, and consequently had to be preserved for compatibility reasons. (It would be bad if, for example, Lotus 1-2-3 stopped working on Windows NT because DDE broadcasts were no longer supported. If the Windows NT team had tried that gambit, nobody would have upgraded and Windows NT wouldn't have survived to make a second version.)

    On the other hand, given the risks involved in DDE broadcasts, you probably would be better off designing your program to not use dynamic data exchange as a data communication mechanism, thereby avoiding the pitfall of message broadcasts. No point contributing to the problem.

  • The Old New Thing

    Why can't I get the pixels of a window that isn't visible on screen?


    If the window isn't visible on the screen, then the pixels simply don't exist.

    The Windows painting model follows the principle of "Don't save anything you can recalculate". Consider: You have a 640x480 display in 16-color mode. That's 150KB of video memory. Take four copies of Notepad and maximize each one. If each of those copies of Notepad required an offscreen cached bitmap, you would be using 450KB of memory just to retain those bits. If your computer has only 640KB, that doesn't leave much memory for anything else.

    Think about what programs need to display. In the vast majority of cases, the programs already need to keep track of the information needed to generate the display anyway, and usually in a more compact form. Notepad, for example, has an edit control which in turn holds the text being edited. Displayed or not, the edit control needs to keep track of the text. (It's not like Notepad would use OCR to read the pixels from the screen and convert them back to text when it needs them!) Any program that has a scroll bar needs to manage information that is not visible on screen. It's hardly a stretch to make it manage information that is visible.

    If you do want to get the pixels from a window that isn't visible on screen, you can try sending it the WM_PRINT message, though this requires that the window you're sending it to even bothers to support the WM_PRINT message. (Many don't.) If you are using Windows XP or later, you can use the PrintWindow function.

  • The Old New Thing

    Why does the CreateProcess function do autocorrection?


    Programs that weren't designed to handle long file names would make mistakes like taking the path to the executable and writing it into the registry, unaware that the path might contain a space that needs quoting. (Spaces—while technically legal—were extremely rare in SFN paths.) The CreateProcess function had to decide whether to "autocorrect" these invalid paths or to let those programs simply stop working.

    This is the battle between pragmatism and purity.

    Purity says, "Let them suffer for their mistake. We're not going to sully our beautiful architecture to accomodate such stupidity." Of course, such an attitude comes with a cost: People aren't going to use your "pure" system if it can't run the programs that they require.

    Put another way, it doesn't matter how great your 1.0 version is if you don't survive long enough to make a 2.0.

    Your choice is between "being pure and unpopular" or "being pragmatic and popular". Look at all the wonderful technologies that died for lack of popularity despite technical superiority. Sony Betamax. Mattel Intellivision. (And, in the United States: The metric system.)

    Electric cars are another example. As great as electric cars are, they never reached any significant market success. Only after conceding to popularity and "sullying" their "purity" by adding a gasoline hybrid engine did they finally gain acceptance.

    I see this happening over and over again. A product team that, hypothetically, makes automated diagramming software, says, "I can't believe we're losing to Z. Sure, Z's diagrams may be fast and snazzy, but ours gets <subtle detail> right, and when you go to <extreme case> their diagrams come out a little distorted, and they're faster only because they don't try to prevent X and Y from overlapping each other in <scenario Q>. We're doing all those things; that's why we're slower, but that's also why we're better. Those people over at Z just don't 'get it'."

    Guess what. People are voting with their wallets, and right now their wallets are saying that Z is better in spite of all those "horrible flaws". Whatever part of "it" they don't get, it's certainly not the "make lots of people so happy that they send you money" part.

  • The Old New Thing

    I'll see (some of) you in Los Angeles in September


    Jeremy Mazner has asked me to put together a 400-level session at this year's PDC. I came up with the title "Five(ish) things every Win32 developer should know (but likely doesn't)". Of course, now I have to think of five things! Here are some ideas I've been kicking around.

    • The memory scene: Physical address space != physical memory != virtual memory != virtual address space
    • Consequences of the way CPUs work: How my O(n) algorithm can run circles around your O(log n) algorithm; why much of what you learned in school simply doesn't matter
    • Parent vs. owner windows
    • Asynchronous input queues, the hazards of attaching thread input, and how it happens without your knowledge
    • Dialog units, DPI and the overloaded term "large fonts"

    Would you go to a talk that covered these topics? If not, what topics would you rather hear me talk about?

    Follow-up: The talk will be a little over an hour. (And fixed the title. Thanks, Dave.)

  • The Old New Thing

    Why does the Run dialog autocorrect but not the Run key?


    In an earlier comment, Aswin Gunawan asked why the Run dialog does autocorrection but not the Run key?

    One is a programmatic interface and the other is an end-user control.

    End users are not expected to be understand how computers do things. They want, and even expect, the computer to help them out with what they're typing. This means spell-checking, guessing missing information, and generally "doing what I mean". For example, web browsers don't require you to type "http://" or "ftp://" in front of web addresses any more; they guess whether you're trying to connect to an FTP site or an HTTP site based on context. Of course, sometimes the computer guesses wrong—maybe you really wanted to use HTTP to connect to a site whose name begins with "ftp."—in which case you can re-enter the command and provide more information so the computer won't have to guess as much.

    Programming interfaces, on the other hand, are for people who do understand how computers do things. If a program makes an invalid request to a function, the expectation is that the function will return an error and not try to "guess" what the programmer "really meant to do". Because a computer program can't look at the "autocorrected" result and say, "No, that's not quite what I meant," and retry the operation with "a few more hints". (Heck, if the program had "some more hints", why not just pass the "fully hinted" version in the first place? Save you the trouble of calling the function twice.)

    Computer programs prefer predictability. Autocorrection and context-driven guessing are unpredictable. Imagine how much harder it would be to write code if a stretch of code changed it behavior based on fuzzy logic like "Well, there was a file in the directory that had a similar name, so I used that one instead", or "This class doesn't have a field called 'current', but the one over there does, so I'll guess that you meant that one."

    I'm sure some people will point out that the CreateProcess function breaks this rule of "don't guess; just fail". We'll discuss this next time.

Page 1 of 4 (35 items) 1234