Windows Services Or Task Scheduler? - Part I

dditweb — Mon, 09 Oct 2006 17:37:00 GMT

Before I even start I’m sure all you know what I’m going to talk about. First let me explain what our team does in Microsoft.

Ours is an automation team whose primary function is develop apps for the Developer Division build lab for managing the build process, track issues, monitor the servers, provide statistics on various parameters . Or in short, basically manage their day-to-day activities. All our apps are web based (for obvious reasons) hosted on multiple web servers connecting to multiple database instances. We use ASP.NET 1.1 & 2.0 with C# and SQL 2005 databases. We have a number of jobs for doing some tasks like updating databases, send notifications to users etc... So for each task we wrote a console app in C# and scheduled via Windows Task Scheduler for various intervals. Roughly we have around 50+ web sites and 70+ scheduled tasks and services.

Some of the problems we faced were

1) Deployment

a. Most of our business logic was embedded in the web sites so we had to distribute the components to the console app as they were hosted in different servers. So every time we made a change to those components we had to make sure that we updated all the servers running our tasks. We minimized this by writing web services in the same web site and calling the web services from the console app. Not a clean solution though.

b. We used a single service account to access our web services and databases. So whenever we created a task in the scheduler we had to provide its credentials. Everytime the password changes we had to manually update all our tasks which is tedious and error prone. Recently we got some script which will update the credentials for all the tasks.

2) Maintenance – As we started creating console app for each task we ended up having 70+ tasks and growing which became a maintenance nightmare. One way to minimize this is by having a single console app with different command line parameters. Truly I’m not crazy about this solution.

3) Error Reporting – There is no way we could get any notifications when a task fails. We had to implement exception handling in each app and notify us via email in case of failure. In some cases we also wanted to know successes to make sure that critical tasks are running fine etc…

4) Access Control – Windows Scheduler requires an admin account to setup various tasks. This is not very practical for our team as we also host some apps for other teams. For security reasons we cannot grant admin access to all of them to our servers. So every time they want to add/modify a task they had to notify us and we did it for them. Also as we added new tasks it became hard for us to differentiate between our teams tasks from them as windows scheduler doesn’t provide any kind of grouping of tasks. So we had to rely on the naming conventions for the tasks.

5) Statistics – For some specialized tasks we wanted to know the duration, return values etc.. Over time which will help us to monitor the tasks and fine tune them if required. This will also be useful to decide if we need to change the run interval or to run in multiple servers etc..

We decided windows services as our solution and wrote a number of windows services one for each associated web site and deployed them. Whenever we wanted a new task we added a timer and executed code in the timer event. Though it solved a few of the above mentioned problems like high frequency tasks, management via mmc etc.. we had to write code for setting and managing the timer, exception handling, reporting etc..

So I set out to improve the situation and find a solution which will get best of the both worlds plus catering much more to the requirements mentioned above. I started finding information from the internet and came across some interesting articles like the ones below

Jon Galloway’s blog – http://weblogs.asp.net/jgalloway/archive/2005/10/24/428303.aspx

Jon says windows services should not be used to run interval based tasks and that windows scheduler should be used instead. But later he agrees that in some cases windows services are more suitable. As I already explained windows scheduler is not suitable for our team.

MSDN Magazine – http://msdn.microsoft.com/msdnmag/issues/05/03/SchedulingASPNETCode/

All these solutions address specific needs but not all of our requirements. So I decided to write a task scheduler from scratch as a windows services using C#. In addition to addressing all the above mentioned problems it should additional features which we will see in a while.

Due to its complexity I decided to develop the service in multiple phases addressing the most critical requirements to nice to haves. (But later we became content with the first few phases and never cared to spend time to develop the rest of the features J. I’ll do when I get time)

Phase I

1) Provide basic task scheduling with intervals based on minutes/hours/daily/weekly/monthly

2) Provide option to have multiple schedules for same task. For. e.g. have a task run at 8am and 8pm everyday and also once at midnight every 1^st day of a month etc..

3) Provide ability to schedule different types of tasks like Web Service, SQL StoredProc, Custom .NET code etc…

4) Allow configurable parameters for each tasks. This could be parameters for the web service, SQL sproc etc.

5) Provide a simple UI to manage the individual tasks.

6) Have ability to enable/disable an individual task, set timeouts etc.. without having to restart the scheduler.

7) Provide basic notifications in case of failures.

8) Provide ability to have multiple task schedulers

The scheduler framework will be implemented as a single class which we named it as DDRTLib.Windows.Services.ServiceBase which derives from System.ServiceProcess.ServiceBase. A developer can simply create a new .NET windows service project and replace the default base class to the this class. No additional code is required at all. This base class defines a single timer which is set to trigger every minute. When the timer event is triggered the framework will loop through all the tasks to be run and spans a new thread from a thread pool to execute the task.

Since we will be defining all our schedules via config file the config section must be registered as a custom section. The whole config section will be handled by a class called DDRTLib.Windows.Services.ScheduledTaskController

Here’s a sample config file

<section name="scheduledTasks"

type="DDRTLib.Windows.Services.ScheduledTaskController, DDRTLib, Version=3.0.0.0, Culture=neutral, PublicKeyToken=6509eb956bdc2c34, Custom=null"/>

</configSections>

<task

name="Daily Summary Mail"

type="Custom"

customType="MyNamespace.CustomTask, MyTasks"

enabled="true"

timeout="5"

</settings>

</task>

<task

name="SanMan Data Cache Refresh"

type="WebService"

webServiceUrl="http://webserver/webservice.asmx"

webMethod="RefreshSanManData"

enabled="false"

timeout="60"

</settings>

</task>

<task

name="TeamStats Queue Health"

type="StoredProcedure"

storedProcName="up_GetDataPointQueueSummary"

connectionString="myconnectionstring"

enabled="true"

timeout="5"

</settings>

</task>

</scheduledTasks>

</configuration>

Note that the above config file does not show the custom sections for the exception management block which our framework uses to publish the exceptions to event log, email etc..

The framework also sets up a file watcher on the config file so that any changes to it will be effected immediately. This allows us to share this config file over network and grant access to appropriate users. If the config file is ill-formed or incomplete the service shuts itself down and notifies the users. If we can also have it notify the users and use previous configuration settings.

We’ve been using this framework for the last 6 months slowly moving all our existing scheduled tasks and services and it works like a charm and saves us from a lot of headache. In my next blog I’ll explain the next phases along with some code excerpts.

Happy Programming J.

Karthik Palanivel

DDITDev : Task Scheduler

Windows Services Or Task Scheduler? - Part I