Larry Osterman's WebLog

Confessions of an Old Fogey
Blog - Title

Applet Mitigations

Applet Mitigations

  • Comments 25

As I've mentioned, applets  can be a plague on your system.  The annoying thing is that it's possible to write applets that aren't so horrible.  And most of the mitigations are really just common sense ideas - there's nothing spectacularly complicated in any of them.

As with the earlier posts in this series, some of the mitigations are common to all types of applets (and all applications in general), and others are specific to various types of applet.

 

Let's start with the basics...

For applets that run all the time:

Do you REALLY need to have an applet running all the time?  The best applet is one that doesn't run at all.  Is it possible to bundle your applet's functionality into an application that the user invokes?  The start menu highlights newly installed applications, so your new application will be visible there, worst case there are other mechanisms for installing your application in locations that are visible to the user (the desktop is one of them (although that has its own set of issues)).

Now that you've decided you need an applet that runs all the time, please reconsider.  Seriously.  I know I just asked you to think about it, but really.  Steve Ballmer says that sometime in 2008, a billion people will be running some version of Windows, that's a LOT of people.  If your product is successful, you're likely to be selling to a couple of million of them - do you believe that your applet provides enough value to every one of those customers that you need to have it running in their face?  Really?

Once you've decided that you REALLY need to have an applet running, make sure that there's a way for the user to turn it off.  There's nothing more annoying than realizing that the software that came with some random piece of hardware that I use maybe once or twice a month is running all the time on my machine.

If you've written an applet because you want to let the user know about some cool feature, why not use the RunOnce key to let the user know about the feature, letting them know how to discover the feature later on, then shut up and never bother them again?

 

For all applets (and all applications that are expected to run all the time):

Think about how to reduce the applets impact on the user.  Reduce the DLL load in your applet whenever possible - each DLL you load consumes a minimum of 4 private pages (16K on x86) and takes between 500K and 1M cycles to load.  Anything you can do to reduce that is better. If you can get away with just loading kernel32.dll and the C runtime library, it's better.  Consider delay loading DLLs that you infrequently use.

Reduce the stack size used for the threads in your applet - by default Windows threads get a 1M commitment and 10K of reserve (which really turns into 12K of reserve due to paging).  That means that every thread is guaranteed to consume at least it's stack commitment space in virtual memory (the good news is that it's virtual memory - normally that'll just be space reserved in the paging file, not real memory).

Reduce the number of processes that your applet needs.  There's rarely a good reason for you to require more than one process to do work.  About the only one I can think of is if you split functionality to increase the amount of code you have running at a high privilege level).  As an example of this, in Windows Vista, the audio stack runs in two separate services - the AudioSrv service and the AudioEndpointBuilder service.  This is because a very small part of the functionality in the audio engine has to do some operations that require LocalSystem access, but the rest of the audio stack can run just fine as LocalService.  So the AudioEndpointBuilder service contains the high privilege code and the AudioSrv service contains the rest.  If you feel you need to have a separate process to provide reliability (you run the code out-of-proc in case it crashes), windows Vista provides a cool new feature called the "restart manager".  The restart manager allows the OS to restart your application if it crashes, reducing the need to run code out-of-proc.

Don't forget that Windows is a multi-user system.  Some of your customers won't want your applet, others will.  So make sure that the settings to enable/disable the applet are instanced on a per-user basis.  It's really annoying when you right click on a notification area icon and see that the "disable this" menu is disabled because you're running as a normal user (which is most users on Vista).  Whenever I see this, I know that the author of the applet didn't consider the normal user case.

If you can target Vista only, consider reducing your thread and I/O priority.  If your applet is performing processing that's not directly related to the user, use the new PROCESS_MODE_BACKGROUND_BEGIN option in the SetPriorityClass API to let the system know that your process should be treated as a low priority background process - that way your applet won't preempt the user's work.  You can also use the new FileIoPriorityHintInfo method of the SetFileInformationByHandle to let the OS to prioritize your I/Os to a handle below that of other user operations.

 

Next: Mitigations for updaters (no post tomorrow since I'm moving offices).

  • On the restart manager: It is almost a no-brainer to not use it. Only Vista? Gee, I cannot imagine any ISV who could afford that...

    "make sure that there's a way for the user to turn it off" That is not going to work. What incentive does a ISV have to provide that option?!? None, really. This is a classical game theory problem. There is a cost to do it the way you suggest it. The benefits of doing it properly are tiny, if you are alone doing it. Plus, any ISV doesn't really get the benefits, rather the platform as a whole gets it, IF everyone does it. Basic economic theory tells you that you will always get the non-cooperative solution in such a situation, you are fighting a battle you cannot win. The way out is to recognise that a) MS is the only party that would really benefit if all ISVs did it properly (simply because the Windows platform would hopefully be known to not have all the crappy negative side effects of applets) and that therefore you guys need to get the incentives right for ISVs. What could that be? Simple, make it even LESS costly for an ISV to provide these options than not to. How could that be achieved? Something like Vista's mobile applet window, maybe with an option to active and deactive per user services or something like that would be a good start. If you provide a prominent UI place for applets to plug into (sidebar might be another place, or even a new notification area), ISVs will want to be there. Then use that and only provide an API to get to that UI space that at the same time allows users to simple enable/disable such services. You can go on an ask ISVs forever to change their behaviour, but unless you understand the economics of this and act accordingly, nothing will change.

  • Davidacoder: Actually it's not that horrible to build stuff for XP and then light up additional functionality on Vista - Office does this, for example.  It does mean additional work, but it's not the end of the world.

  • davidacoder makes a good point. It is about economics.

    There are many ISVs that work hard to make something difficult to disable, kill, change.

    Think all the crap^H^H^H^H applications that every time they run they hijack file formats, add themselves to autostart, to system try, to desktop, start menu, quick launch, explorer context menus, etc.

    This is a lot of work. They do it because *they want* to be "in your face," not because is easy.

  • Speaking of the audio service, don't forget AudioDG.exe.  On my system it's got a working set of 40 megs.

    Honestly, if a 'craplet' is taking up 4 megs or so of memory, I think (de-sensitized by all of the components in Vista), 'Oh well'.

  • If audiodg is taking up that much memory, you should check to see if an updated driver is available for your audio solution (or disable all system effects).  We've had reports of buggy audio drivers that have memory leaks causing this - on most machines I've looked at, audiodg consumes around 4M of RAM.

  • But if an ISV cares about reliability, it will care about reliability on Vista AND old Windowsversions. Following your suggestion would mean that the ISV should use Vista's restart manager on Vista systems, and a homegrown solution on older versions. What is the incentive for an ISV to do that? The homegrown version has to be developed in any case, what does he gain from expending more dev time to also implement use of Vista's restart manager? Why would he use it? There is no incentive at all in such a situation.

  • davidacoder That most likely means that MS should make restart manager available downlevel (i.e. on XP, since that is the only other platforms [yes I mean plural] that matter since every thing else is more or less EOL) Now the economic question is does MS benifit enough to do that in a way that is good for customers?

  • charless, that ain't gonna happen.  

    There's no reason that an application can't take advantage of the restart manager when running on Vista - Office somehow manages to take advantage of the restart manager when it's running on Vista, and on previous OS versions, it doesn't.

    When you run on Vista, new features and functionality light up, on previous OS releases, they don't.

  • Larry, independent developers are in a different situation than Office. Unlike Microsoft, we don't common up-level bosses that say we must take advantage of the new Vista features to prevent the company from looking like hypocrites. Our bosses look at the huge XP installed base and relatively slow uptake of Vista and say, "Make sure it works on XP and Vista, but only add Vista-specific features if it buys *us* something."

    It's often easy to leverage Vista's low-priority I/O with a few lines of code, but it's another to completely retool the crapplet design and installation process to use the Vista scheduler or other functionality. That takes time, and time takes money, and time prevents shipping.

    I really like the Vista scheduler, it's an undiscovered gem. To be realistic it wouldn't even help if Microsoft back-ported it to XP at this point unless they also allowed third parties to ship it as a redist with their products since most systems wouldn't have it installed. By the time all that happened it will be 2011 and XP will be fading fast.

  • Dave, actually that's not <i>quite</i> the way it worked - the reality is that our "common up-level boss" was Bill Gates (Office is in Jeff Raikes org, Windows was in Jim Allchin's org, now in Kevin Johnson's org).

    As I understand it (obviously, I'm not on the Office team, neither am I on the restart manager team), the Office team learned about the new features we were planning on Vista (the same way that our other customers did), realized that they were specifically designed to address certain pain points that customers (including Office) were feeling, and adopted them.

    It didn't add a significant amount of complexity to their test matrix (they already had to test on Vista, this just meant that their Vista tests cases also included testing the restart functionality).

    I'm trying to be very clear about the Vista-specific functionality in these discussions.  The vast majority of the stuff I'm talking about works on OS's as far back as Windows ME (some of the advice goes way back to NT 3.1).

    Even if you don't adopt the Vista-specific functionality, the other recommendations are still valid.

  • Does anyone know of any important third party app that takes advantage of some new technology that is in Vista, i.e. lights it up as Office does it? Or has that happened when XP was introduced? Just curious, I honestly don't know.  But none comes to mind, right now. I no major up besides Office does it, then that would suggest to me that their using it was a incentive that came from being in the same company with Windows :)

  • Here's another rule I'd like added to your list - Be Hibernate-Aware. When I wake my notebook up from sleeping after an hour or so, there's no problem. But if it's been sleeping for a couple days, say, over the weekend, it takes a very long time to finally wake up due to all the hard drive activity. I had to disable the page file in fact to keep it from taking 3-5 minutes to wake up.

    I'm not 100% sure of the problem but it appears to be that every app in the system is doing its "hey, it's been a while since I've done task x" routines. So maybe, Explorer is re-checking the Start Menu to figure out what to highlight, the Recycle Bin is clearing out files, every updater type applet is busy trying to get on the network to check for their updates, Google Desktop is refreshing its Gmail index, etc. All of which starts to happen instantly as soon as the machine starts up the processes again after waking up. Which causes a huge amount of hard drive activity, not good for a 4200 rpm notebook hard drive. Especially when all you want to do is check email real quick.

    Disabling the page file helped quite a bit for me. I tried killing as many applets as possible, keeping my drive and pagefile defragged, but nothing solved the 3-5 minute wake-up time until I killed the page file.

    Simple rule to be hibernate-aware: if it's been > 60 seconds since your last update loop iteration, then the machine has probably been hibernated. Wait a random amount of time from 20 seconds to 5 minutes before doing whatever checks unless it's absolutely necessary to do it immediately (like perhaps logging into IM). Auto-updaters do not fall into this category.

  • Scott, that's an interesting point.

    I don't see why turning your paging file off helps the hibernate case, all you're doing is forcing the system to keep all your code in memory (which means that the stuff you're not using hurts the stuff that you are using).

    If you're running XP, I suspect you're seeing what we call "standby list erosion" (slava oaks talks about it here: http://blogs.msdn.com/slavao/archive/2005/01/29/363181.aspx#1730358, the good news is that Vista has logic to alleviate that problem.

  • > (no post tomorrow since I'm moving offices).

    (1) You mean the applets will let you move offices *before* they run?

    (2) You need to switch back from C# to C++.  Then you can just update a pointer instead of moving.

  • Norman, I wish that moving offices was as easy as redirecting a pointer.  It's a royal pain in the neck (you see, I have these extremely large and distressingly fragile lego models that have to be moved by hand)...

    Btw, I do all my real work in C++ :)

    Btw2: Daniel just opened up his new Lenovo x61 tablet (a present for taking honors algebra 2 over the summer), I was literally astonished at the amount of stuff that came preloaded on it (norton's security suite, diskkeeper, office 12 (trial edition), the list was quite long).

Page 1 of 2 (25 items) 12