Larry Osterman's WebLog

Confessions of an Old Fogey
Blog - Title

Applet Mitigations - Services.

Applet Mitigations - Services.

  • Comments 22

As a senior developer at Microsoft, you often find yourself participating on a number of v-teams.  One of the v-teams I'm on is responsible for approving new services added to Windows.  As I've mentioned before, I'm a nutcase about stuff running on my machines, and services are absolutely among the things I care about passionately.  As a part of my work on that v-team, I wrote this little bit up a couple of years ago (it's been edited slightly to remove proprietary information):

I've been sitting watching the <new services v-team> process for a couple of months now, and I've seen a number of trends that concern me.

Every single new feature (and it seems like there have been thousands of new features) seems to require its own service to perform operations.  Don't get me wrong - it's wonderful that these functions are running as services and not as separate processes.

But every single one of the new services that I see being requested is enabled on all SKUs of Windows.  All of them, it seems.  And they're all auto-start.

The <new services v-team> has done a terrific job of reducing the number of own-process services that are running.  That's truly awesome, and it's great for our customers.

But I don't think that they're going far enough.  We need to take a harder line on our services.  Because even if multiple services are hosted in a single process, they each still burns at least one thread.  And that thread consumes working set.  And it affects startup time.  And if your code has memory (or GDI/User object) leaks, it can render computers unusable.  The other thing to consider is that every running service in Windows increases the Windows attack surface. 

In Windows XP, we had 40ish services on a running system.  We've got almost twice that on a default Vista install these days (assuming my test machine is a default longhorn install).[1]

Now I appreciate that everyone's feature is critical for their customers, but I'm wondering if they're all necessary for all customers.  Do you REALLY believe that your code is going to be used by every single one of the nearly a billion users of Windows?  Is your service going to make every single one of those billion people's lives better?  If your service isn't, then maybe every one of those billion people don't need to be running your code.

As I've said, I've been thinking about this for a while, and I think I've got a few things that should be considered when you're trying to figure out if your service really needs to be installed.

First off,  I know that your feature is the most important thing you're doing, but that's true for every single one of the developers working on the Windows product.  We can't all be number one, so think very seriously about the relative importance of your feature.

If your service is auto-start, is it REALLY necessary?  Will every user of Windows achieve positive benefits from your service?

If your service is tied to a piece of hardware, does your service need to be running if the hardware isn't present?  Can you tie the service to the installer for your hardware?

If your service is tied to a particular UI, and the user never invokes your UI, is your service doing the user any good?  Can your UI start the service if it's not running?

Does your service REALLY need to be enabled and auto-start (even auto-start-delay) on every SKU?  Really?

How is your feature/service discoverable?  If your feature isn't easily discoverable, does the service that supports that feature really have to be run until the user discovers your feature and starts to use it?

Now for some services this is clearly the case.  But for a huge number of the services that we've been coming up with, it's equally clearly not.

Even my own service, Windows Audio doesn't meet all of these criteria.  I'd be more than willing to have the service be manual start unless there's an audio card present, and to change the installer for audio adapters to enable the service.  Because on a machine without audio hardware, there's no point in the service running until the hardware arrives.  There IS one important scenario where it's important to have the Windows Audio service running: that's Remote Desktop - when running a remote desktop, even if the server doesn't have audio hardware, we can still play audio using the TS client's audio hardware.[2]

But that's a relatively weak scenario.  And I'd be willing to change it (or work to change the remote desktop service to ensure that the audio service is started when a client connects).  Are you?

This all is a bit of a digression - it's not about mitigations, it's about the hard decisions you should make when thinking about adding services, but it's worth publishing anyway.

So how do you mitigate services?  First off, combine like services into a single process.  That way, instead of taking two processes, you only consume one process (see my earlier post where I listed the costs of a process). 

Secondly, as I indicated above, consider making your service a manual start service that's triggered by some UI action.  Unless there's a real need for your service to be running all the time, let the UI (or an API if your service surfaces an API) start your service.

Third, seriously consider making your service a delayed auto-start service - this is functionality new in Vista/Windows Server 2K8 that allows the service controller to delay starting your service so it doesn't interfere with boot time[3].

In addition, seriously consider how much time you spend in your service's start routine.  The less time the better (especially if you're an auto-start service).  The less work you can do before reporting that your service has started to the service controller, the faster the system will boot.

 

Tomorrow: Applet best practices - collecting the thoughts of the previous several posts into a single post.

[1] Please note: While there ARE more services in Vista than in XP, this comment is mostly hyperbole.

[2] This didn't happen, the powers that be decided that since every workstation class machine with a Vista logo had to have an audio solution that it was ok to keep the audio service as an auto-start service (they also felt that audio was going to be used by every one of those users :)

[3] Yeah, I know - it's a vista-only mitigation, but it's a good one.

  • PingBack from http://msdnrss.thecoderblogs.com/2007/08/22/applet-mitigations-services/

  • In previous articles, I&#39;ve pointed out: Programmer Hubris - He&#39;s just not that into you Programmer

  • Excellent article Larry. I'm often looking in the startup registry to delete unnecessary autoruns e.g. Adobe Acrobat Reader!!!

  • Re: Item [2]

    What about Longhorn Server?  I've got hundreds of servers, none of them has sound, and only a handful are used for Terminal Services.

  • 2K8 You had me there, as an electronics engeneer my brain assumed you were talking about windows 2800 (as 2K8 means that in electronics, usually for measuring resistors), took me a few seconds to switch to 2008 :) damn

    Tnx for the great series :)

  • Combining services in a single process has some very bad consequences too. Consider the recent pervasive problem with Automatic Updates on XP and Windows Installer (the one where AU, when using Microsoft Update, would crash when checking for updates if any updates for Windows Installer products (such as Office) were available).

    I've personally treated multiple machines with this problem. In each case the same thing: when AU crashed, it took down the svchost.exe it was running in. The result: audio died, the theming service died, so did the computer browser service, wireless zero config, and a bunch of other stuff.

    That really didn't make for a very good eXPerience.

  • I am the same way after every patch day or so I have to go check what they turned back on, same thing with every install, that groove stuff in office is really one of the worst, really dislike. Now I do agree there are some services I use but not all the time. So since they do not stop and start those services when the app starts I set tham all to manual and then I have my batch files that stop and start the services on the rare occasion I need to use an app that uses that service. Being a developer my primary machine must have about 100 services in there. Of which maybe 20 are set to automatically start up. the rest have either been disabled or set to manual. However Users that do not know how to do this really just take the perf hit and continue on grumbling about how slow windows is.

  • Multiple services in a single process got me a real headache a couple of months ago. Something was using 99% of the CPU whenever I'd leave the computer alone and not touch the mouse. Touching the mouse gave me a minute or so to investigate what was causing this. Of course - Task Manager gave me the answer - svchost.exe ;)

    Using Process Explorer from Sysinternals I nailed it down to about 20 or so services it could be (the svchost instance). But how would you continue from here to nail it down to the exact process that's taking up all the CPU?

    I think it was Windows Update, but I was never sure. Problem is gone now anyways.

  • August, there's an easier way.  Enumerate the services in the svchost instance (tasklist /svcs will do that, as will taskmgr in Vista).  Then, from the command line, issue the command "sc config <servicename> /type= own".  To put it back into the shared svchost instance, issue the same command line with /type= shared.  I wrote a post about this a couple of years ago.

  • But you should be careful that the two shared services aren't doing something that relies on them being in the same process. They probably shouldn't, but then again, developers do a lot of things that they shouldn't!

  • A related issue to service proliferation is the mistaken idea that a long task can be made painless by making it run in the background. Two recent examples are Visual Studio's Intellisense and the .NET Framework's background pre-JITter, both of which have been known to run for minutes at 100% CPU. Making a task run in the background prolongs the pain and, contrary to popular belief, a thread that is set to low priority can still have major system impact: it consumes memory, it shortens battery life on laptops, and it can burn significant disk bandwidth. Worse yet, such background tasks don't always provide a readily available UI for "Stop Now" when they are causing problems.

    Another concern I have over excessive services and background work is that I believe it has a security impact, because people get used to computers doing random and unexplained things on their own, and don't question when the suddenly the disk starts running, downloads occur, and random windows pop up. Heck, with the driver installation hacks some have done, it might not be surprising anymore if the mouse pointer starts moving by itself.

  • Alun, the only two services I know of that do that are the browser and the server service.  I'm not aware of any others.

  • >> Does your service REALLY need to be enabled and auto-start (even auto-start-delay) on every SKU?  Really?

    We use services since some of our features require admin access. Unfortunately we need them to be running all the time since non-admin users need the feature too - yet they can't just start the service when they need the feature (since non-admins can't start and stop services)...

    Is there a way around this?

  • Matthew, unfortunately no :(, unless you're willing to grant interactive users the right to start your service, which means that you need to consider the threats associated with that.

  • A case in point is the Windows Media Player Network Sharing Service (WMPNetwk.exe and WMPNSCFG.exe) which is installed by default with WMP11. Unfortunately it keeps running even after turning off media sharing from the UI.

Page 1 of 2 (22 items) 12