By default, if you have some long running system service, you should target it to run in services.exe.  To quote the services.exe white paper  when it described why putting say a web server in device.exe was a bad idea:

However, using device.exe also posed some problems. A lot of other components run in device.exe, including the network stacks and real device drivers, which could make tracking resource leaks very difficult. If device.exe is leaking memory and there are 50 other device drivers running, how does a person find the guilty party? Even worse, a poorly written DLL could potentially corrupt memory in the TCP/IP stack, PCMCIA driver or other device driver, causing inexplicable crashes.

Services.exe was created to deal with these issues. Services.exe is designed to have a programming model and application programming interface (API) set that are similar to device.exe to facilitate moving device drivers that weren't really device drivers (think about a Web server) from device.exe to services.exe. If you're not familiar with device.exe, don't worry. This article starts from the basics. If you're familiar with device drivers on Windows CE, much of this article will cover familiar ground.

As the author of services.exe, it pains me to say that sometimes device.exe has its uses.  In particular, you really need to run in device.exe if:

1) You need to touch physical hardware.  In current versions of WinCE you probably can get away doing this in services.exe.  In the next version of CE this will no longer be an option, so do the right thing and get out of services.exe now.  In any event it's not exactly clean to be touching hardware in a process named services.exe when there's a device.exe designed for this.

2) Related to (1), you need your driver/"service" to take xxx_PowerUp and xxx_PowerDown notifications.  Unfortunately some sample services implement xxx_PowerUp/xxx_PowerDown and may make you think that a service can receive this.  Services.exe itself is notified when power notifications come in but it will not pass these on to your service for various reasons.  Device.exe will pass these on.

3) You need direct access to something else that lives in device.exe.  A firewall component that examines every incoming IP packet as it arrives is a good example.  While this isn't touching hardware, it does need to talk to the TCP/IP stack.  Having each packet go across process to be examined would kill performance, which is why the firewall is loaded by device.exe.

4) Something in device.exe needs your component to be up and running before it can proceed.  At boottime, all device drivers are started up before the first service is.

A good (or not so good) example of where Microsoft has really screwed this one up is with the notification API service.  Notification API basically lets apps register to be notified when certain system events happen or when some time event occurs.  Since there's no hardware interaction, seems this should live in services.exe, right?  Wrong.  It turns out a ton of stuff in device.exe needs this API too.  These drivers have to go into a lame polling mode to make sure that the notification service has started already.

As a semi-random side note, in the next CE version we are going to do the right thing here in notification API.  In the interim, if you run into this problem yourself and you're an OEM you can set BSP_NOTIFY_IN_DEVICE=1 and run makeimg again to have notifications put back into device.  This isn't an officially tested scenario mind you, but I've had customers on the newsgroup do this and have never heard them complain.

[Author: John Spaith]